Family members and friends of the author voice deepfake

What happened

A cybersecurity journalist conducted an experiment to test the viability of AI-driven voice scams by attempting to impersonate themselves to five friends and family members. The author gathered source audio from their own interviews as a podcast host and recorded new samples, deliberately compressing them into .WAV files to simulate the quality of audio typically harvested from social media platforms like TikTok or Instagram. Using a script generated by Claude that emphasized urgency and panic, the author tested several free, open-source, and standalone AI voice-cloning tools, including ElevenLabs, Kikivoice.ai, Voicebox, and IndexTTS.

The author found that while some tools produced high-quality audio, they often lacked the necessary emotional nuance or sounded too professional for a distress-based scam. Ultimately, the author utilized the Chatterbox model within Voicebox for the calls. To mimic a real-world attack, the author used a temporary phone number to contact the five subjects. The results showed that four of the five participants immediately identified the call as fake, citing indicators such as the flat tone of the voice, strange intonation, mispronounced words, and the use of an unrecognized phone number. One participant, who was of an older generation and hard of hearing, believed the caller was the author. The experiment highlighted that while current AI voice-cloning technology still struggles with emotional authenticity, it is becoming increasingly accessible, requiring as little as 10 seconds of audio to create a potentially passable impersonation.