The online platform X has caused major anger after individuals employed its artificial intelligence assistant Grok to alter images of females by removing their clothing.
Numerous cases have been seen where the system has been employed to virtually strip female subjects to render them in bikinis without consent, as well as to put them in sexualized situations.
Developer Response and Legal Stance
XAI, the firm behind Grok, remained silent to inquiries, except for an standard message stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to outlaw nudification tools. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a prison sentence and significant fines".
"The platform has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "assess the danger" of people viewing illegal content on their networks, but did not confirm whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' requests when they tag it in a message.
It is often used to provide commentary or additional context to other posters' comments, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been condemned for allowing users to produce images and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating inappropriate content of public figures.
"X has been enabling the production and spread of these images for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of individuals in a graphic fashion".
In a comment, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said networks such as X were obligated to take "suitable steps" to "lessen the danger" of users encountering illegal content on their services and to take down it swiftly when they are notified of it.
