The social media platform X has caused significant backlash after people utilized its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Numerous cases have surfaced where the system has been used to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to put them in inappropriate contexts.
Developer Response and Regulatory Stance
XAI, the firm behind Grok, remained silent to questions, except for an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to outlaw image stripping software. Under a new law, anyone who provided such technology would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was at this time investigating X or Grok in regarding AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for functions—which answers X users' prompts when they reference it in a tweet.
It is often used to offer commentary or more background to other users' remarks, but people on X are also capable of alter an posted image through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been criticised for allowing users to produce images and clips with nudity and sexually suggestive content, and it was earlier accused of generating graphic content of public figures.
"The platform has been enabling the production and sharing of these pictures for a long time without taking any steps and we have not witnessed any intervention by regulators," the expert added.
XAI's own usage rules bans "portraying likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or share non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said platforms such as X were obligated to take "appropriate actions" to "lessen the danger" of users viewing illegal content on their platforms and to remove it promptly when they become aware of it.
