The problem AI content moderation cannot solve - Rest of…

A woman was subjected to a prolonged campaign of abuse when her ex-husband manipulated everyday photos of her to damage her personal and professional life. Despite the victim reaching out for assistance, platform policies failed to provide protection because the images were not considered explicit, highlighting a significant gap in how technology companies address non-consensual image-based abuse.

What happened

The victim was targeted by her ex-husband, who used manipulated versions of her own selfies—showing her fully clothed with friends or displaying personal grooming choices—to harass her. He distributed these altered images to her WhatsApp contacts, intending to harm her reputation at her workplace and create conflict within her family, specifically targeting her relationships with her brothers and parents. When the victim sought help from WhatsApp and local officials, she was told that the images did not constitute explicit content, and therefore, no action could be taken against the perpetrator. This case illustrates the limitations of current moderation frameworks, which often rely on Western definitions of intimacy and explicit content. Because the abuse involved non-nude, everyday imagery, the victim remained unprotected despite the clear intent to cause harm. This incident underscores the broader issue where technology companies fail to account for the lack of consent in image manipulation, prioritizing existing content policies over the actual harm experienced by users.