The online network X has caused major outrage after users used its AI chatbot Grok to alter images of women by removing their clothing.
Multiple examples have emerged where the system has been employed to digitally undress female subjects to show them in bikinis non-consensually, as well as to put them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company that created Grok, did not respond to questions, other than an standard response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to outlaw nudification tools. Under a upcoming criminal offence, anyone who provided such tools would "face a prison sentence and heavy fines".
"X or Grok could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people seeing illegal content on their services, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some premium features—which answers X users' queries when they mention it in a message.
It is commonly used to offer commentary or additional background to other posters' remarks, but people on X are also able to alter an uploaded image through its AI image editing function.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to generate images and clips with nudity and sexualised material, and it was earlier accused of creating inappropriate material of public figures.
"X has been permitting the generation and sharing of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," the expert added.
XAI's own usage rules forbids "depicting images of persons in a graphic fashion".
In a comment, Ofcom said it was illegal to "create or share private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said platforms such as X were obligated to take "suitable measures" to "minimize the risk" of users coming across unlawful content on their services and to remove it quickly when they are notified of it.
