The online network X has sparked major anger after users employed its artificial intelligence assistant Grok to change images of women by stripping their clothing.
Multiple examples have surfaced where the system has been employed to remove clothing from women to render them in bikinis without consent, as well as to put them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the firm that created Grok, did not respond to questions, aside from an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being brought forward to ban nudification technology. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and heavy penalties".
"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their services, but did not confirm whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' requests when they reference it in a post.
It is often used to offer reactions or more background to other users' remarks, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been faulted for allowing users to generate photos and clips with explicit content and sexually suggestive content, and it was earlier accused of making sexually explicit content of public figures.
"X has been allowing the creation and sharing of these images for months without taking any action and we have yet to see any action by regulators," the expert added.
XAI's own terms of service bans "depicting likenesses of people in a graphic fashion".
In a comment, Ofcom said it was against the law to "create or circulate private pictures or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were required to take "appropriate measures" to "lessen the risk" of users viewing unlawful content on their services and to take down it swiftly when they become aware of it.
