Detect Deepfakesby Resemble AI
Deepfake case study · Audio

A red teamer voice deepfake — Aug 2024

During testing of OpenAI's GPT-4o system, the Advanced Voice Mode unexpectedly imitated a tester's voice, saying "No!" The model synthesized the sound from.

Incident date
Aug 9, 2024
Target
a red teamer (adversarial tester)
Updated May 6, 2026 · 1 min read

During testing of OpenAI's GPT-4o system, the Advanced Voice Mode unexpectedly imitated a tester's voice, saying "No!" The model synthesized the sound from its training data. Safeguards have since been implemented.

Sources