The social media network X has caused significant outrage after individuals utilized its AI chatbot Grok to manipulate pictures of women by taking off their garments.
Multiple cases have been seen where the tool has been used to remove clothing from female subjects to make them appear in bikinis without permission, as well as to put them in inappropriate contexts.
Company Reaction and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, other than an standard message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to ban nudification technology. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and heavy penalties".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some premium features—which answers X users' queries when they reference it in a message.
It is often used to offer reactions or additional context to other posters' remarks, but people on X are also can modify an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The tool has been faulted for enabling users to create images and clips with explicit content and sexualised material, and it was previously accused of generating inappropriate content of celebrities.
"X has been enabling the generation and sharing of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and stated this covered explicit AI images made with AI.
It said networks such as X were obligated to take "appropriate measures" to "lessen the risk" of users coming across illegal content on their platforms and to remove it promptly when they learn of it.
