AI Deepfake Ads Put Meta’s Scam-Prevention Model Under Pressure

Introduction

AI-generated faces and voices are changing the economics of online fraud. Investor Mark Cuban says he regularly hears from people who believe he sells a $294 diabetes treatment. According to Cuban, the advertisements use an AI copy of his appearance and voice, yet still appear on Meta’s platforms.

Key points

• Deepfakes make false endorsements more convincing. Scammers can borrow a public figure’s identity to promote medical products, investments or other offers. A disclosure saying that a clip was AI-generated or intended as entertainment may do little to undo the effect of the sales pitch.
• Meta’s internal estimates suggest a major business conflict. Reuters reported that Meta documents estimated around $16 billion, or about 10% of 2024 revenue, could come from ads linked to scams and prohibited products. Another document described roughly 15 billion “higher risk” ads reaching users each day, with that category valued at about $7 billion annually in a late-2024 estimate.
• Automated enforcement can leave a gray zone. The documents say Meta’s systems must assess an advertiser as at least 95% likely to be running a scam before banning it. Accounts below that threshold may remain active and compensate for higher scrutiny by spending more on ads.
• Meta disputes the interpretation. Company spokesperson Andy Stone said the documents present a selective and distorted picture of Meta’s anti-fraud work. He called the 10% estimate rough and overly broad, while also saying scam-ad complaints had fallen 58% over the previous 18 months and that more than 134 million scam ads were removed in 2025.

Why it matters

The dispute is about more than a few fraudulent videos. It raises a structural question: what should a platform do with advertising that looks suspicious but does not yet meet its removal threshold? Conventional scams often rely on clumsy text or images. Generative AI can reproduce a person’s identity, delivery and emotional cues at low cost, making quick visual judgments less reliable.

Older adults are particularly exposed. The FBI said people aged 60 and above reported $7.7 billion in internet-crime losses in 2025. Investment scams accounted for the largest share of those losses, while AI appeared in more than 3,100 complaints involving more than $352 million in reported losses. Those figures cover reported cases only, so they do not capture victims who stay silent.

Platforms may need to go beyond counting removals after an ad has already circulated. Stronger advertiser verification, evidence requirements for medical and financial claims, and systems that connect repeated creative variations could reduce the damage earlier. Users should independently search a product name alongside terms such as “scam,” and report suspicious ads promptly.

AI has made impersonation faster and cheaper. Meta’s response will ultimately be judged not only by the number of ads it removes, but by how transparently it explains its thresholds and how effectively it prevents known patterns from returning.

Source: Hacker News