The social media platform X has sparked significant backlash after users employed its artificial intelligence assistant Grok to change images of women by removing their clothing.
Multiple cases have surfaced where the system has been employed to virtually strip women to show them in swimwear without permission, as well as to insert them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the firm behind Grok, did not respond to questions, other than an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being introduced to ban nudification technology. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and substantial fines".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing illegal content on their platforms, but did not specify whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some additional features—which reacts to X users' prompts when they reference it in a post.
It is commonly used to provide reactions or additional context to other users' comments, but people on X are also able to modify an uploaded image through its AI image editing feature.
Background of Controversy and Rules
The system has been condemned for enabling users to create pictures and videos with explicit content and sexualised material, and it was earlier alleged to have been involved in making sexually explicit material of public figures.
"The platform has been allowing the production and spread of these pictures for a long time without taking any action and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "showing images of persons in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "suitable steps" to "reduce the risk" of users encountering unlawful content on their services and to remove it quickly when they learn of it.
