The online network X has caused significant anger after individuals employed its AI chatbot Grok to alter photos of females by stripping their garments.
Several instances have surfaced where the tool has been used to virtually strip female subjects to make them appear in bikinis without permission, as well as to insert them in sexually explicit contexts.
Developer Response and Regulatory Position
XAI, the organization behind Grok, offered no comment to questions, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification software. Under a proposed law, anyone who supplied such technology would "receive a jail term and heavy penalties".
"The platform is capable of halting these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "consider the danger" of people viewing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some additional features—which reacts to X users' prompts when they mention it in a post.
It is commonly used to give reactions or more background to other posters' remarks, but people on X are also can modify an uploaded image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The system has been criticised for enabling users to create pictures and clips with nudity and sexually suggestive material, and it was in the past accused of creating graphic content of celebrities.
"X has been enabling the generation and spread of these images for months without taking any measures and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service forbids "showing likenesses of persons in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or child sexual abuse material" and confirmed this included sexual deepfakes created with AI.
It said services such as X were required to take "appropriate measures" to "minimize the risk" of users viewing illegal content on their platforms and to remove it quickly when they learn of it.
