Insight from Yoshi Soornack, Project Flux
How platform-scale AI deployment reveals governance patterns that matter
Grok, the AI chatbot built into Elon Musk’s X platform, spent the first week of January at the centre of an international conversation about AI governance. By the time X restricted the feature to paying subscribers on 9 January, governments across four continents had raised concerns, regulators had opened enquiries and researchers were documenting what many called industrial-scale content moderation challenges.
This is about understanding what happens when powerful AI capabilities move faster than governance frameworks can constrain them and what that pattern means for organisations deploying AI tools in their own environments.
Project professionals should pay attention, because the underlying lessons apply directly to programme delivery.
Understanding What Happened
X added an image generation feature to Grok last summer, including capabilities designed to generate varied content. Unlike other mainstream AI models, Grok is built into one of the most popular social media platforms. Users can prompt it privately or tag it publicly in posts, and Grok responds in the open.
In late December, users discovered they could upload photos of real people and command Grok to modify them. The most common requests involved placing subjects in revealing clothing or suggestive scenarios. Some images appeared to depict minors.
Genevieve Oh, a social media researcher, conducted a 24-hour analysis of images the Grok account posted to X. She found that Grok was producing roughly 6,700 images per hour during peak periods. By comparison, the five other leading websites for similar content averaged 79 new images hourly during the same period.
The scale matters. Unlike standalone websites, Grok has built-in distribution through X’s social media infrastructure. Images generated by the chatbot appeared directly in user feeds, often without consent from the people depicted. Many individuals reported concerns, and some said their reports went unanswered whilst images remained live on the platform.
Ashley St. Clair, who became a high-profile case, shared her experience with Fortune.
“When I saw [the images], I immediately replied and tagged Grok and said I don’t consent to this. [Grok] noted that I don’t consent to these images being produced…and then it continued producing the images, and they only got more explicit.” Source: Fortune
The Governance Lessons That Transfer
Here’s what makes this situation instructive for project professionals: the patterns reveal governance gaps that appear across AI deployments.
First, safety resources must match deployment scale. According to sources familiar with the situation at xAI, the company’s safety team was small compared to competitors. In the weeks leading up to the controversy, three key staffers left: Vincent Stark, head of product safety; Norman Mu, who led the post-training and reasoning safety team; and Alex Chen, who led personality and model behaviour post training.
Second, leadership priorities fundamentally shape outcomes. Multiple sources reported that concerns were raised internally about inappropriate content, but those concerns did not translate into effective preventive measures at the scale needed.
Third, commercial incentives require careful balancing with safety requirements. When X finally restricted image generation, it made the feature available only to paying subscribers rather than removing it entirely.
Fourth, policy enforcement must match capability deployment timing. Grok’s own acceptable use policy prohibits “depicting likenesses of persons in a pornographic manner” and “the sexualisation or exploitation of children.” Those policies existed. The challenge was enforcement at the point where users interacted with the AI.
The Pattern Project Teams Should Understand
X’s decision to restrict advanced features to paying users has two interpretations. The company is balancing innovation with commercial sustainability, and premium tiers fund ongoing development. At the same time, critics argue that monetising access to powerful capabilities without proportionate moderation creates challenges for people affected by misuse.
Some see this as a test case for how platforms manage generative AI features: whether to prioritise broad access or implement stricter controls to manage potential harms.
For project professionals, the lesson transcends any specific platform. The lesson is about how quickly powerful AI capabilities can move from novel to operational and what governance looks like when it works versus when it doesn’t.
Project teams will see stakeholders adopting generative tools at accelerating rates. That adoption creates both opportunity and considerations around appropriate use. The challenge is building governance frameworks that enable beneficial use whilst managing legitimate concerns about misapplication, accuracy and accountability.
Shadow AI use is already widespread across organisations. Most project teams have limited visibility into which AI tools people use or how they use them. ChatGPT, Claude, Gemini and other models are embedded in workflows because they are fast, accessible and often genuinely helpful. Adding generative capabilities for images and video amplifies both the value and the governance considerations.
What This Means for Project Delivery
Organisations deploying AI tools without governance frameworks appropriate to the capability risk creating the conditions for similar challenges, just at different scales and in different contexts.
Consider how this plays out in project environments. A team member uses an AI tool to generate images for a client presentation. The images might be based on copyrighted material or might depict stakeholders inappropriately. The client raises concerns. The question then becomes: who owns accountability? The team member? The project manager? The organisation? The AI provider?
Or an AI tool generates content that conflicts with data protection obligations, contractual confidentiality or regulatory disclosure requirements. The issue only becomes apparent later, after decisions have been made. What controls should have prevented that outcome?
The Grok situation demonstrates what happens when capability deployment outpaces governance development. Project professionals can learn from this pattern without experiencing it directly.
Practical Steps for Building Governance
Building effective AI governance requires five focused actions:
- Acknowledge that people are already using generative tools for project work. Comprehensive prohibition is not viable. Detection after the fact is insufficient. The effective response is transparent guidance that defines appropriate use, required reviews and escalation processes.
- Create specific guidance rather than generic statements. Concrete examples work better than broad principles. Can AI-generated images appear in client presentations? Under what review process? Can AI-generated text appear in formal project documentation? Who validates accuracy?
- Establish review processes proportionate to stakes. This does not mean reviewing every AI interaction. It means defining trigger points. High-stakes communications, formal deliverables and client-facing materials should undergo human review before reaching stakeholders.
- Build awareness of how AI tools affect documentation, communications and stakeholder expectations. AI can accelerate valuable work, but it also introduces new considerations. Teams benefit from understanding what can go wrong and how to prevent issues proactively.
- Clarify accountability structures. When an AI tool contributes to a project output, who is responsible for that output? The person who prompted the AI? The project manager who approved its use? The organisation that provided or failed to restrict access?
The Broader Context
xAI faced inquiries from France, India, Malaysia, the UK and the European Union. The European Commission ordered X to retain all internal documents and data related to Grok until the end of 2026. Ofcom, the UK’s media regulator, made urgent contact with X and xAI to assess compliance with legal duties. US lawmakers raised concerns as well.
Simultaneously, while the platform was experiencing these challenges, X’s leaders noted that engagement metrics were reaching record levels. Meanwhile, xAI was announcing its $20 billion funding round.
This contrast reveals an important dynamic. AI capabilities are advancing rapidly, capital is flowing into AI development at unprecedented levels, and regulatory frameworks are working to keep pace. Organisations are adopting tools faster than they are building appropriate governance.
Project professionals operate in this environment. The opportunity is to build governance proactively rather than reactively.
The Governance Opportunity
The Grok situation is instructive rather than exceptional. It demonstrates patterns that will appear across AI deployments at different scales and in different contexts. AI tools introduce capabilities that traditional governance frameworks were not designed to manage.
Organisations that recognise this early can build appropriate controls. Organisations that wait will build them through experience that could have been avoided.
The organisations that capture advantage from AI whilst managing it responsibly will:
- Build governance frameworks before incidents force them
- Create clear guidance that enables beneficial use
- Establish review processes proportionate to stakes
- Develop accountability structures for AI-mediated work
- Build organisational capability in AI governance progressively
This is happening across sectors right now. Financial services organisations are defining how AI can inform investment decisions. Healthcare providers are establishing protocols for AI-assisted diagnosis. Manufacturing firms are creating frameworks for AI-controlled production systems.
