Weeks after Elon Musk’s X was flooded with AI-generated images depicting people, including children, in sexualized ways without consent, California is investigating how the hell it happened. The state’s Attorney General Rob Bonta announced Wednesday that he is opening a probe into the situation to determine if X and xAI, Musk’s AI company and the maker of the chatbot Grok that was used to generate the pornographic images, broke the law.
“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet,” Bonta said in a statement. He also urged xAI to take “immediate action” to ensure that type of content can’t be created and spread.
Bonta seems to have quite a bit of public support for the investigation. A recent YouGov poll found that a whopping 97% of respondents said that AI tools shouldn’t be allowed to generate sexually explicit content of children, and 96% said those tools shouldn’t be capable of “undressing” minors in images.
The investigation will focus on the trend that cropped up on X over the winter holiday, which saw users prompting Grok on the platform to modify images of people to show them in various states of undress. The trend got big enough that, according to AI content analysis firm Copyleaks, Grok was generating a nonconsensually sexualized image every minute. Some of those images included children, which users prompted Grok to undress and depict in underwear or bikinis. Often, users requested that Grok add “donut glaze” to the faces of the subjects of the images.
Musk—the CEO of both X, the company where the images were being shared, and xAI, the company that makes the AI model used to generate the images—has opted to obfuscate or claim ignorance of the situation. In a post made prior to California’s investigation being announced, Musk said, “I not aware of any naked underage images generated by Grok. Literally zero.”
The narrowness of his statement does a lot of heavy lifting, saying he’s unaware of any “naked underage images.” That doesn’t refute the existence of naked images, images of undressed underage people, or people being depicted in sexualized situations. Nor does it address the fact that many of those images were nonconsensual, generated without the permission of the person being depicted. In countless cases, the imagery has been directly used to harass accounts on X.
To the extent that Musk was willing to admit that such a problem is even possible, he said it’s the fault of the users, not the AI model or platform spreading the content. “Obviously, Grok does not spontaneously generate images; it does so only according to user requests. When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state,” he said. “There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately.”
That’s about in line with the sparse response that X has offered to the situation. In a post from X Safety, the company said, “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content,” but took no responsibility for enabling it. For what it’s worth, Musk was also mockingly reposted content created as part of the trend, including AI-generated images of a toaster and a rocket in a bikini.
California is the first state in the country to launch an investigation into the situation. Authorities in other countries, including France, Ireland, the United Kingdom, and India, have all started looking into the nonconsensual sexual images generated by Grok and may also bring charges against X and xAI. The Take It Down Act, which was passed into law last year, doesn’t require platforms like X to create notice and removal systems for nonconsensual images until May 19, 2026.
Read the full article here
