Research map / Detection and segmentation / Open-set and few-shot detection
DF40: Toward Next-Generation Deepfake Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DF40, a large benchmark of 40 deepfake techniques (face-swapping, reenactment, entire face synthesis, editing) with four cross-forgery/cross-domain/open-set protocols and eight detectors, yielding findings on generalization gaps.
Problem
Deepfake detectors that look strong under the common FF++-train/CDF-test protocol fail to generalize to diverse, modern deepfake types, because existing datasets cover few forgery methods, use outdated techniques, and evaluate on narrow types.
Approach
A large, diverse dataset spanning four deepfake categories (FS, FR, EFS, FE) with 40 methods, plus four standardized evaluation protocols (cross-forgery, cross-domain, unknown-forgery/domain open-set, one-verse-all) to reveal detector generalization failures.
Key results
- CLIP-large trained on DF40 (FF) reaches 0.898 average AUC on unknown-domain testing, versus 0.644 for Xception.
- Training on FS (FF) gives about 0.8 AUC on EFS, while training on EFS gives only about 0.6 on FS; Xception trained on FS (FF) drops to 0.657 on FR (CDF) and 0.642 on EFS (CDF).
- CLIP-large trained only on face data achieves 0.746 average AUC on GenImage non-face deepfakes, versus 0.535 for Xception.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CLIP instead of Xception | Deepfake detection trained on face data · GenImage non-face domain | AUC: 0.535 -> 0.746 | Helped |
| Use CLIP instead of Xception | Deepfake detection · DF40 (FF), average across unknown face deepfakes | AUC: 0.644 -> 0.898 | Helped |
| Train on FS rather than EFS | Deepfake detection models · Cross-forgery evaluation on FF domain | AUC | Helped |
| Replace Xception with an SoTA detector (SRM, SPSL, RECCE, or RFM) | Deepfake detection · DF40 evaluation scenarios | AUC | No clear effect |
| Change both forgery type and data domain | Xception trained on FS (FF) · FR (CDF) and EFS (CDF) | AUC: 0.657 and 0.642 | Hurt |
| Apply GFPGAN super-resolution to SRI (ablation) | FSGAN self-reconstruction images · SRI | AUC: 0.772 -> 0.983 | Helped |
| Apply GFPGAN super-resolution to SRI (ablation) | BlendFace self-reconstruction images · SRI | AUC: 0.835 -> 0.825 | No clear effect |
| Apply GFPGAN super-resolution to SRI (ablation) | LIA self-reconstruction images · SRI | AUC: 0.746 -> 0.988 | Helped |
| Apply GFPGAN super-resolution to SRI (ablation) | Wav2Lip self-reconstruction images · SRI | AUC: 0.564 -> 0.833 | Helped |
| Apply GFPGAN super-resolution to SRI (ablation) | DiT self-reconstruction images · SRI | AUC: 0.687 -> 0.997 | Helped |
| Apply GFPGAN super-resolution to SRI (ablation) | DDIM self-reconstruction images · SRI | AUC: 0.713 -> 0.946 | Helped |
| Apply GFPGAN super-resolution to SRI (ablation) | e4e self-reconstruction images · SRI | AUC: 0.543 -> 0.978 | Helped |
Surprising findings
- SoTA detectors (SRM, SPSL, RECCE, RFM) show no clear advantage over the simple Xception baseline across forgeries.
- Super-resolution applied to fake images raises detection AUC by over 20 points for several EFS and FE methods, suggesting SR artifacts are detected.
Limitations
- Limited analysis of video-level detectors; video model results are only in the appendix.
- Some test-only methods (e.g., HeyGen, DeepFaceLab) have small sample sizes (50 and 100) due to their cost.
- No error bars; fixed seed used to match DeepfakeBench.
Experimental setup
Datasets: FaceForensics++ Celeb-DF CelebA FFHQ VFHQ UADFV GenImage LRS3
Benchmarks: DF40 GenImage
Metrics: AUC
Baselines: Xception CLIP SRM SPSL RECCE RFM SBI
Builds on: DeepfakeBench FaceForensics++ Celeb-DF SBI
Closest papers by meaning
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.91)
- Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection (AAAI 2024 · similarity 0.90)
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion (NeurIPS 2024 · similarity 0.90)
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.90)
- Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector (CVPR 2025 · similarity 0.90)
- Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.