Research map / Label-efficient and robust learning / OOD and anomaly detection
D^3: Scaling Up Deepfake Detection by Learning from Discrepancy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D3, a deepfake detector that pairs CLIP features of an image and its patch-shuffled version with self-attention, and scales deepfake detection from train-on-one to train-on-many generators.
Problem
Deepfake detectors trained on a single generator generalize poorly when scaled to training on multiple generators; existing methods either underfit in-domain data or sacrifice ID fit for OOD generalization.
Approach
A frozen CLIP ViT-L/14 extracts features from the original image and a patch-shuffled copy; self-attention over the two features learns shared artifacts from their discrepancy, with a linear classifier on top.
Key results
- D3 reaches 90.7% total mean accuracy and 96.8% AP, vs 86.6%-ish UFD baseline; OOD mean accuracy 87.4% vs UFD 81.4% (+5.3%, the paper's headline figure).
- Patch shuffling outperforms horizontal flip, vertical flip, and random rotation as the discrepancy operation (total mean acc 91.3% vs 84.6% for horizontal flip).
- DIRE gets 97.6% ID accuracy but only 68.4% OOD when trained on 8 generators.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use D3 instead of UFD | Scaled-up deepfake detection · Scaled-up test set, out-of-domain | mean accuracy: 0.814 -> 0.867 | Helped |
| Add a patch-shuffled image branch to the original-image branch (ablation) | D3 · Validation set, ablation groups 2 and 4 | total mean accuracy: 0.858 -> 0.913 | Helped |
| Add self-attention to the original and patch-shuffled features (ablation) | D3 · Validation set, ablation groups 3 and 4 | total mean accuracy: 0.868 -> 0.913 | Helped |
| Replace the patch-shuffled branch with a second original-image branch (ablation) | D3 · Validation set, total | total mean accuracy: 0.913 -> 0.853 | Hurt |
| Use shuffled images in both branches rather than original and shuffled images (ablation) | D3 · Validation set, total | total mean accuracy: 0.913 -> 0.780 | Hurt |
| Replace patch shuffling with horizontal flipping (ablation) | D3 · Validation set, total | total mean accuracy: 0.913 -> 0.846 | Hurt |
| Replace patch shuffling with vertical flipping (ablation) | D3 · Validation set, total | total mean accuracy: 0.913 -> 0.886 | Hurt |
| Replace patch shuffling with random rotation (ablation) | D3 · Validation set, total | total mean accuracy: 0.913 -> 0.891 | Hurt |
Surprising findings
- Horizontal flipping as a discrepancy signal does not beat the one-branch baseline, since flipped features are nearly identical to the original's.
Limitations
- The two-branch design runs the vision backbone twice, doubling computation relative to UFD.
- The required large-scale merged dataset may not fully meet the proposed setup's needs.
- Collecting a high-quality detection dataset is expensive.
Experimental setup
Datasets: UniversalFakeDetect GenImage
Benchmarks: UniversalFakeDetect GenImage
Metrics: mean accuracy average precision global AP
Baselines: CNNDet Patchfor LNP DIRE UFD
Builds on: UniversalFakeDetect (UFD) CLIP DIRE Patchfor
Closest papers by meaning
- Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection (AAAI 2025 · similarity 0.89)
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.89)
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.88)
- Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning (ICLR 2026 · similarity 0.88)
- Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning (AAAI 2024 · similarity 0.88)
- Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.