Research map / Image generation / Editing and style transfer
DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffusionFake, a plug-and-play framework that injects deepfake-detector features into a frozen Stable Diffusion 1.5 via a ControlNet-style guide module to reconstruct source and target images, improving cross-dataset generalization without inference overhead.
Problem
Face forgery detectors generalize poorly to unseen manipulation domains, since they overfit to method-specific artifacts.
Approach
Invert the deepfake generation process: deepfakes mix source and target identity information, so the detector's encoder features are filtered into source- and target-related components and used to guide a frozen Stable Diffusion U-Net to reconstruct the source and target images; only the encoder and classifier are kept at inference.
Key results
- With EfficientNet-B4 backbone, DiffusionFake raises Celeb-DF frame-level AUC from 73.51 to 83.17 and average AUC from 75.80 to 81.88 in cross-database evaluation trained on FF++.
- With ViT-B backbone, DiffusionFake achieves the best average AUC of 83.78 and EER of 25.17 across the five unseen test sets.
- Ablation on Celeb-DF: removing the pre-trained SD weights drops AUC from 83.17 to 77.35 (with filter and weight modules), and removing the filter module drops it to 78.67 range for the SD-with-weight-only variant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DiffusionFake to EfficientNet-B4 | EfficientNet-B4 · Celeb-DF | AUC: 73.51 -> 83.17 | Helped |
| Add DiffusionFake to ViT-B | ViT-B · Wild Deepfake | AUC: 75.46 -> 80.14 | Helped |
| Add DiffusionFake to EfficientNet-B4 | EfficientNet-B4 · DFDC-P | AUC: 70.51 -> 77.35 | Helped |
| Add DiffusionFake to EfficientNet-B4 | EfficientNet-B4 · DiffSwap | AUC: 77.38 -> 82.02 | Helped |
| Add pretrained Stable Diffusion weights (ablation) | DiffusionFake without pretrained Stable Diffusion · Celeb-DF | AUC: 71.87 -> 73.87 | Helped |
| Add Feature Filter and Weight modules (ablation) | DiffusionFake with Stable Diffusion · Celeb-DF | AUC: 77.35 -> 83.17 | Helped |
| Add Weight module (ablation) | DiffusionFake with Stable Diffusion and Feature Filter · Celeb-DF | AUC: 80.79 -> 83.17 | Helped |
| Add Feature Filter module (ablation) | DiffusionFake with Stable Diffusion and Weight · Celeb-DF | AUC: 78.67 -> 83.17 | Helped |
| Add DiffusionFake to ResNet | ResNet · Celeb-DF | AUC: 68.89 -> 75.27 | Helped |
| Add DiffusionFake to EfficientNet-B0 | EfficientNet-B0 · Celeb-DF | AUC: 71.74 -> 76.31 | Helped |
| Add DiffusionFake to ViT-Small | ViT-Small · Celeb-DF | AUC: 70.59 -> 74.58 | Helped |
Surprising findings
- Without pre-trained Stable Diffusion weights, guided reconstruction gives a much smaller gain, indicating the generative prior is essential.
Limitations
- Requires paired source and target images for training, which may not be available in real-world settings.
- Effectiveness against multi-source or partial manipulations is not established.
- The method could potentially be misused as an adversarial discriminator to create harder-to-detect forgeries.
Experimental setup
Datasets: FaceForensics++ Celeb-DF DFDC Preview Wild Deepfake DeepFake Detection (DFD) DiffSwap
Benchmarks: Celeb-DF Wild Deepfake DFDC-P DFD DiffSwap FaceForensics++
Metrics: AUC EER ACC
Baselines: Xception Face X-ray F3-Net MAT GFF LTW LRL DCL PCL+I2G SBI UIA-ViT RECCE
Builds on: ControlNet Latent Diffusion Model DDPM ICT UCF
Closest papers by meaning
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.91)
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.90)
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.90)
- FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge (NeurIPS 2024 · similarity 0.89)
- FakeDiffer: Distributional Disparity Learning on Differentiated Reconstruction for Face Forgery Detection (AAAI 2025 · similarity 0.89)
- Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.