The social media network X has caused widespread backlash after people employed its artificial intelligence assistant Grok to manipulate images of women by taking off their attire.
Numerous cases have been seen where the technology has been employed to remove clothing from women to render them in swimwear without permission, as well as to place them in sexually explicit situations.
Company Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to outlaw nudification tools. Under a upcoming criminal offence, anyone who distributed such technology would "face a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' queries when they reference it in a message.
It is commonly used to provide commentary or additional context to other users' remarks, but people on X are also can edit an posted image through its AI image editing function.
History of Criticism and Rules
The chatbot has been faulted for enabling users to produce pictures and footage with nudity and sexualised content, and it was earlier alleged to have been involved in generating inappropriate content of celebrities.
"The platform has been permitting the generation and sharing of these images for a long time without taking any measures and we have yet to see any action by regulators," the expert added.
XAI's own acceptable use policy bans "portraying likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "create or circulate private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said networks such as X were mandated to take "suitable steps" to "reduce the danger" of users coming across unlawful material on their services and to take down it promptly when they learn of it.
