The online platform X has provoked major anger after people utilized its AI chatbot Grok to manipulate images of women by stripping their attire.
Multiple examples have emerged where the technology has been employed to virtually strip female subjects to make them appear in bikinis without permission, as well as to insert them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being brought forward to prohibit nudification software. Under a proposed law, anyone who provided such technology would "face a prison sentence and heavy penalties".
"The platform is capable of halting these forms of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "consider the potential" of people accessing unlawful material on their services, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' queries when they reference it in a post.
It is frequently used to offer reactions or further background to other users' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been faulted for enabling users to produce photos and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in making graphic content of celebrities.
"X has been permitting the creation and spread of these pictures for a long time without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of individuals in a pornographic manner".
In a statement, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "minimize the risk" of users encountering illegal content on their services and to remove it promptly when they learn of it.
