Wildlife biologist with a passion for sloth conservation and sustainable ecosystems.
The social media platform X has provoked widespread outrage after individuals used its AI chatbot Grok to change photos of females by stripping their attire.
Multiple instances have surfaced where the tool has been employed to remove clothing from women to show them in swimwear without permission, as well as to put them in sexually explicit contexts.
XAI, the company behind Grok, remained silent to requests for comment, except for an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to outlaw nudification tools. Under a upcoming law, anyone who distributed such tools would "receive a jail term and substantial penalties".
"The platform is capable of halting these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
The regulator Ofcom said tech firms must "assess the danger" of people accessing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' requests when they tag it in a post.
It is commonly used to give commentary or additional context to other posters' remarks, but people on X are also can edit an uploaded photo through its AI image editing function.
The system has been condemned for permitting users to create photos and footage with explicit content and sexualised material, and it was earlier accused of creating inappropriate material of public figures.
"The platform has been enabling the creation and sharing of these images for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or child sexual abuse material" and confirmed this included explicit AI images created with AI.
It said platforms such as X were obligated to take "necessary measures" to "lessen the risk" of users viewing illegal content on their services and to remove it swiftly when they are notified of it.
Wildlife biologist with a passion for sloth conservation and sustainable ecosystems.