my gf thinks this video is real, but it looks clearly AI…

A concerning new trend on TikTok involves the use of AI-generated personas to hijack the content of real creators. These deepfake accounts replicate the exact speech patterns of original videos, including natural stumbles and fillers, to push viral narratives or disinformation under the guise of an authentic speaker. TikTok creator Ali Palmer, known as @aliunfiltered_, reported that her own content regarding a news story was stolen and re-created by an AI-powered account that used her exact words. While traditional content theft involves reposting original footage, this emerging tactic uses synthetic media to create a new, realistic-looking digital avatar that mimics the original creator’s script. Forensic experts have confirmed that these personas are generated using current AI tools, often resulting in low-quality deepfakes that nonetheless succeed in generating significant engagement.

What happened

The tactic involves identifying viral TikTok content and having an AI-generated persona recite the original creator's script word-for-word. Forensic media experts, including Hany Farid of UC Berkeley, analyzed multiple accounts and confirmed the use of AI to generate both the visual persona and the audio. These accounts often maintain a consistent visual style but exhibit inconsistencies in voice register and quality across different videos. Darren Linvill of Clemson University noted that the most significant indicator of this deception is the variation in voice characteristics when comparing multiple clips from the same account. Despite the accounts accumulating millions of views by targeting viral topics ranging from conspiracy theories to tabloid stories, the motivation behind these operations remains unclear. While TikTok policies require the labeling of AI-generated content, these accounts often bypass such measures until identified by external investigations. The practice raises significant privacy concerns for creators whose identities and intellectual property are being co-opted to drive artificial virality.