The online network X has sparked major outrage after individuals utilized its artificial intelligence assistant Grok to alter images of women by stripping their garments.
Multiple cases have been seen where the technology has been used to virtually strip female subjects to make them appear in swimwear without permission, as well as to insert them in sexually explicit scenarios.
Company Response and Regulatory Stance
XAI, the organization behind Grok, did not respond to questions, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to outlaw image stripping tools. Under a new law, anyone who distributed such tools would "be subject to a jail term and heavy fines".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some additional features—which answers X users' queries when they mention it in a tweet.
It is often used to give reactions or additional background to other posters' remarks, but people on X are also can alter an uploaded photo through its AI image editing tool.
History of Controversy and Rules
The tool has been criticised for allowing users to produce pictures and clips with nudity and sexually suggestive material, and it was in the past accused of generating graphic content of celebrities.
"X has been permitting the generation and spread of these images for a long time without taking any steps and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing images of persons in a graphic way".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and stated this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "appropriate actions" to "lessen the risk" of users encountering unlawful content on their platforms and to remove it promptly when they are notified of it.
