U.K. woman felt ‘violated, assaulted’ by deepfake Grok images

Workers install lighting on an "X" sign atop the company headquarters, formerly known as Twitter, in downtown San Francisco, July 28, 2023. (AP Photo/Noah Berger)

CARDIFF, United Kingdom -- British academic Daisy Dixon felt “violated” after the Grok chatbot on Elon Musk’s X social media platform allowed users to generate sexualized images of her in a bikini or lingerie.

She was doubly shocked to see Grok even complied with one user’s request to depict her “swollen pregnant” wearing a bikini and a wedding ring.

“Someone has hijacked your digital body,” the philosophy lecturer at Cardiff University told AFP, adding it was an “assault” and “extreme misogyny.”

As the images proliferated “I had ... this sort of desire to hide myself,” the 36-year-old academic said, adding now “that fear has been more replaced with rage.”

The revelation that X’s Grok AI tool allowed users to generate images of people in underwear via simple prompts triggered a wave of outrage and revulsion.

Several countries responded by blocking the chatbot after a flood of lewd deepfakes exploded online.

According to research published Thursday by the Center for Countering Digital Hate (CCDH), a nonprofit watchdog, Grok generated an estimated three million sexualized images of women and children in a matter of days.

CCDH’s report estimated that Grok generated this volume of photorealistic images over an 11-day period -- an average rate of 190 per minute.

After days of furore, Musk backed down and agreed to geoblock the function in countries where creating such images is illegal, although it was not immediately clear where the tool would be restricted.

“I’m happy with the overall progress that has been made,” said Dixon, who has more than 34,000 followers on X and is active on social media.

But she added: “This should never have happened at all.”

She first noticed artificially generated images of herself on X in December. Users took a few photos she had posted in gym gear and a bikini and used Grok to manipulate them.

Under the U.K.’s new Data Act, which came into force this month, creating or sharing non-consensual deepfakes is a criminal offence.

‘Minimal attire’

The first images were quite tame -- changing hair or makeup -- but they “really escalated” to become sexualized, said Dixon.

Users instructed Grok to put her in a thong, enlarge her hips and make her pose “sluttier.”

“And then Grok would generate the image,” said Dixon, author of an upcoming book “Depraved,” about dangerous art.

In the worst case, a user asked to depict her in a “rape factory” -- although Grok did not comply.

Grok on X automatically posts generated images, so she saw many in the comments on her page.

This public posting carries “higher risk of direct harassment than private ‘nudification apps,’” said Paul Bouchaud, lead researcher for Paris non-profit AI Forensics.

In a report released this month, he looked at 20,000 images generated by Grok, finding over half showed people in “minimal attire,” almost all women.

Grok has “contributed significantly to the surge in non-consensual intimate imagery because of its popularity,” said Hany Farid, co-founder of GetReal Security and a professor at the University of California, Berkeley.

He slammed X’s “half measures” in response, telling AFP they are “being easily circumvented.”

By Anna Malpas and Justine Gerardy, AFP