A coordinated legal offensive is underway against xAI, Elon Musk’s artificial intelligence company, as numerous US state attorneys general (AGs) move to restrict the misuse of its chatbot, Grok. The action stems from widespread reports of users exploiting Grok to generate millions of explicit, including child sexual abuse material (CSAM), images and videos earlier this year.
Regulatory Response and Investigations
At least 37 US state and territorial attorneys general are involved, with a bipartisan group of 35 publishing an open letter demanding immediate action. The letter highlights that Grok’s capabilities have been exploited to create non-consensual intimate deepfakes and sexualized content targeting women and children.
Investigations have already begun in states like Arizona, where Attorney General Kris Mayes launched a probe on January 15, citing the disturbing nature of the generated imagery. California’s Attorney General, Rob Bonta, issued a cease-and-desist letter to xAI, demanding the removal of CSAM and non-consensual content. Florida’s Attorney General’s Office is also in discussions with X to ensure child safety protections are in place.
The Scale of the Problem
Recent estimates indicate that Grok users generated around 3 million photorealistic sexualized images in just eleven days, including approximately 23,000 depicting children. Unlike X, Grok’s website initially lacked age verification measures, making it easier to access harmful content. xAI’s response to the allegations was dismissive, claiming that reporting was “Legacy Media Lies.”
Age Verification Laws and Enforcement
The legal pressure coincides with the implementation of age verification laws in several states, requiring proof of age for accessing pornography. However, enforcement is complicated by the sheer volume of AI-generated content and the difficulty in determining what constitutes pornography. Some states, like Louisiana, require more than one-third of a site’s content to be considered explicit before restrictions apply.
Future Legislation and Tech Industry Response
State lawmakers are now considering further legislation to address AI-generated CSAM. Arizona state representative Nick Kupper has proposed a bill mandating age verification for performers in AI-generated content, while Georgia’s senate majority leader Jason Anavitarte plans to introduce legislation to criminalize the creation of AI-generated sexual material involving minors.
Pornhub, a major adult content platform, has blocked itself in many states with age verification laws, arguing that the requirements are impractical and invasive. The company proposes device-based age verification as a potential solution, but tech giants like Google, Apple, and Microsoft have yet to respond to the suggestion.
The swift regulatory response underscores growing alarm over the potential for AI-generated abuse material, and signals a new era of legal scrutiny for tech companies failing to address the risks of their own technologies.
