Back to articles

TwinD Tackles the Gap Between Face Swaps and Fully Synthesized Faces

3 min read

Why face-swap detectors can fall behind

Many facial deepfake detectors are trained on a familiar kind of manipulation: a generated face is inserted into an otherwise real frame. That process can leave clues around the blend boundary or in local image texture. Newer systems can instead synthesize the whole face, removing the obvious seam that detectors may have learned to recognize. The paper introduces TwinD as an approach to narrowing this gap between blending-based fakes and entire face synthesis (EFS).

Two kinds of training “twins”

TwinD uses fakes from FaceForensics++ alongside two additional types of examples constructed from real frames. The first, a reconstruction twin, is made by passing a real frame through an autoencoder and then restoring the original pixels in a randomly selected region. The intended signal is the trace left by a neural decoder elsewhere in the reconstructed image.

The second, a blob twin, combines an altered copy of a real frame with the original frame. It is designed to expose the detector to blending artifacts without requiring a particular face generator or a mask shaped around a face. The two examples target different clues: reconstruction-related traces and artifacts introduced when image regions are mixed. In principle, that gives the detector signals that are not tied to one named manipulation tool or one specific face boundary.

The paper also reports a result for training with twins alone: without using a deepfake dataset, single models achieved an average AUROC of 94.3% across three seeds. This suggests that the way training examples are constructed may provide useful supervision even when examples of known deepfakes are unavailable. It does not, by itself, show that a detector can identify every future kind of generated content.

Reported benchmark results and what they mean

The evaluation covers 19 datasets released between 2019 and 2026. According to the paper’s summary, the strongest prior facial deepfake detector averaged 96.2% AUROC on ten blending datasets, while none of the prior detectors exceeded 78.1% on five EFS datasets. General-purpose AI-generated-content detectors did not close that gap. TwinD reports a mean AUROC of 96.3% overall, with 98.1% on the blending datasets and 95.6% on the EFS datasets. AUROC measures how well a system separates real from manipulated examples; higher values generally indicate stronger discrimination.

The notable point is not just the small increase over the earlier overall score, but the reported improvement on full-face synthesis. If replicated across sources and deployment conditions, training on complementary artificial cues could help detectors adapt as forgery methods change. Still, a benchmark score cannot establish performance on every generator, heavily compressed video, or real-world distribution. The abstract does not provide details such as error rates for individual manipulation types. TwinD is therefore evidence for a promising generalization strategy, not proof that deepfake detection is solved. The authors say they have released the code, model, and training data.

Source: Hugging Face Daily Papers

Comments

Checking sign-in status...

Loading comments...

Related articles