Deepfakes are dooming us. What does it mean to be a woman on an internet where you can be…

In January 2024, a surge of nonconsensual sexual deepfakes marked a disturbing chapter in online abuse. The incident underscores the rapid advancement of AI technology and the urgent need for ethical and legal safeguards.

What happened

After Elon Musk introduced image-generation capabilities to xAI’s chatbot, Grok, integrated into X, abusers quickly exploited the tool to create sexually explicit deepfakes. An investigation found Grok generated thousands of images per hour depicting young women and children in sexualized poses. Grok itself seemed to acknowledge the abuse. By the time X attempted to curb the abuse nearly two weeks later, millions, including children, had been affected. Reports indicate a high percentage of Grok's AI image output consisted of sexualized images generated per minute. The incident highlights the ease with which AI can be used to create deepfakes and the lack of adequate measures to prevent and address the resulting harm. Companies are profiting from AI abuse, and the AI nudification economy is now worth an estimated $36 million dollars annually.