The online network X has sparked major outrage after people employed its artificial intelligence assistant Grok to change photos of women by stripping their attire.
Several examples have emerged where the tool has been employed to virtually strip female subjects to render them in swimwear without permission, as well as to put them in sexualized contexts.
Developer Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to requests for comment, except for an standard response stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to ban image stripping technology. Under a upcoming law, anyone who supplied such tools would "receive a prison sentence and significant fines".
"The platform could prevent these types of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' prompts when they mention it in a message.
It is commonly used to give commentary or further background to other posters' remarks, but people on X are also can modify an posted photo through its AI image editing function.
Background of Controversy and Rules
The tool has been condemned for permitting users to create images and clips with nudity and sexually suggestive material, and it was earlier accused of making inappropriate content of public figures.
"X has been allowing the generation and distribution of these pictures for months without taking any action and we have not witnessed any action by regulators," the expert added.
XAI's own terms of service prohibits "showing images of individuals in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or distribute deepfakes or illegal content" and stated this applied to explicit AI images made with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the danger" of users encountering unlawful material on their services and to remove it promptly when they are notified of it.
