Canada says xAI’s Grok broke privacy law over Deepfake images - The American Bazaar

Canada’s federal privacy watchdog has determined that Elon Musk’s xAI violated national privacy laws by releasing an image-generation tool for its Grok chatbot without implementing adequate safety measures. The investigation, which began in January 2026, concluded that the platform enabled users to generate and distribute non-consensual sexualized deepfake imagery of real individuals.

What happened

The investigation, led by Privacy Commissioner Philippe Dufresne, found that the tool lacked sufficient protections to prevent the creation of harmful, AI-generated content. Users were able to alter photographs of real people without their consent, leading to the production of sexualized deepfakes. This regulatory finding highlights significant deficiencies in the platform's initial safety architecture.

While the Canadian commissioner lacks the authority to impose direct financial penalties or force specific policy changes, the ruling serves as a major critique of xAI's product deployment. In response to the scrutiny, xAI has implemented new restrictions aimed at preventing the editing of images involving real individuals in inappropriate or revealing contexts. The company has also pledged to move toward a model of proactive content monitoring rather than relying solely on a reactive, complaint-based system.

This incident is part of a broader wave of international regulatory attention regarding generative AI. Authorities in the United Kingdom, Italy, and other jurisdictions have initiated their own investigations into Grok’s capabilities and its adherence to data-protection and privacy laws. As governments globally struggle to manage the risks associated with non-consensual sexual content and misinformation, this Canadian ruling increases the pressure on AI developers to prioritize safety guardrails alongside rapid technological innovation.