The social media network X has caused significant backlash after users utilized its AI chatbot Grok to manipulate images of females by removing their garments.
Several examples have emerged where the tool has been used to virtually strip female subjects to show them in bikinis without permission, as well as to place them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the firm responsible for Grok, offered no comment to inquiries, other than an standard message stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to ban nudification software. Under a proposed law, anyone who supplied such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for functions—which responds to X users' requests when they mention it in a message.
It is commonly used to provide commentary or more context to other posters' remarks, but people on X are also can edit an uploaded image through its AI image editing feature.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to generate pictures and videos with nudity and sexually suggestive content, and it was previously alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been permitting the production and distribution of these pictures for months without taking any steps and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said services such as X were mandated to take "necessary measures" to "minimize the risk" of users coming across unlawful content on their services and to take down it swiftly when they are notified of it.
