Meta Ads Exposed a Safety Gap Around AI Nudification Tools

Introduction

Meta platforms recently carried advertisements for Kromix, an AI image and video service presented as an “AI image styler.” The campaign promoted an unrestricted experience and suggested that users could place real people into sexual scenarios. In one video, a woman closely resembling a prominent female US politician appeared beside US flags before the clip cut to a pornographic scene involving a woman with the same face. The ads were removed after journalists and researchers asked Meta about them.

The episode is another indication that harmful deepfake services are not merely circulating in obscure online communities. They are also testing the advertising, app-store, and payment systems of major technology platforms.

Key points

• Kromix invited paying users to upload photographs and offered scenarios that included “bedroom rape” and other sexually explicit themes.
• The advertisements were directed exclusively at male users and used the tagline “Unleash Next-gen AI Magic.” Some videos displayed beaches, mountains, or waterfalls as cover images to disguise the material that followed.
• Meta’s ad library showed 32 ads connected to the campaign. Individual ads ran for between five and 46 hours, while most received no more than 10 impressions.
• Meta said it had already removed hundreds of thousands of ads promoting nudification services and was improving automated detection, while acknowledging that no system is perfect.
• Apple said Kromix appeared to have added prohibited content and features after its initial review and removed the app from the App Store.

How the ads slipped through

Meta’s rules prohibit sexual imagery in advertising and ban the promotion of nudify and “AI kissing” apps. Yet the review process relies primarily on automated systems. Advertisers can exploit newly created pages, low-volume test campaigns, coded language, and harmless thumbnails to reduce the chance of detection. The page connected to this campaign had been created recently and had no followers, signals that could have justified additional scrutiny.

The risk is also difficult to capture by examining an ad alone. The most harmful capability may appear only after a user installs the linked app and uploads a photograph. A system that checks the visible frame, caption, or landing page may therefore miss what the product actually enables. Earlier research also found Meta ads involving sexualized minors and face-swapping tools, suggesting that the problem extends beyond adult nudification services.

Why it matters

Nonconsensual intimate imagery can damage a person’s reputation, employment, privacy, and physical safety. Female politicians are particularly exposed because official portraits, press-conference footage, and campaign videos are widely available. Once low-code generation tools turn ordinary public media into sexualized material, online abuse can be produced at much greater scale and used to intimidate or discredit women in public life.

For platforms, removing individual ads is not enough. Effective enforcement requires coordinated checks across advertisements, app features, developer accounts, updates, and distribution channels. Meta says it shares information about removed apps and developers with other technology companies, but this case shows the limits of relying on automated review followed by reactive removal. The central challenge is shifting from detecting an obviously explicit image to identifying an entire harmful service and the tactics used to conceal it.

Source: Ars Technica AI