The social media network X has caused major anger after users employed its AI chatbot Grok to manipulate pictures of females by taking off their attire.
Numerous instances have been seen where the technology has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the firm that created Grok, offered no comment to requests for comment, aside from an auto-reply response stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification tools. Under a proposed law, anyone who distributed such technology would "receive a jail term and heavy penalties".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing illegal content on their platforms, but did not confirm whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which responds to X users' prompts when they reference it in a tweet.
It is often used to provide commentary or further background to other users' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification feature.
History of Controversy and Rules
The chatbot has been condemned for allowing users to produce images and clips with explicit content and sexualised material, and it was earlier accused of creating sexually explicit content of celebrities.
"The platform has been enabling the generation and spread of these images for months without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was against the law to "make or share deepfakes or illegal content" and stated this included explicit AI images generated with AI.
It said services such as X were mandated to take "suitable steps" to "minimize the risk" of users encountering unlawful content on their platforms and to remove it quickly when they learn of it.
