Research map / Image generation / Autoregressive and latent generation
From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FIA-USA, a pseudo-fake augmentation that simulates face-inconsistency and up-sampling artifacts, combined with AFFS feature selection and RCR contrastive regularization, to train an EfficientNet-B4 deepfake detector that generalizes across face forgery datasets.
Problem
Building a universal deepfake detector that generalizes to unseen facial manipulations whose forgery artifacts vary widely across generators.
Approach
Categorize deepfake artifacts into two shared types, Face Inconsistency Artifacts (FIA) and Up-Sampling Artifacts (USA), and synthesize pseudo-fakes that contain only these artifacts via multi-type masks blending a target face reconstructed by AE or super-resolution models, plus an automatic channel-selection step and region-aware contrastive loss.
Key results
- Frame-level cross-dataset average AUC of 0.852 when trained on FF++ c23, with 0.867 on CelebDF-v2 and 0.901 on CelebDF-v1.
- Average AUC of 87.8% on six DF40 face-swapping types, reported as surpassing RECCE by 9.7% and SBI by 23.4%.
- Video-level AUC of 0.941 on CelebDF-v2 and 0.866 on DFDCP, averaged 0.904 with RCR in the appendix comparison.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FIA-USA instead of the reported baseline detectors | EfficientNet-B4 deepfake detector · DiFF T2I, I2I, FS, and FE | frame-level AUC: 80.20 -> 86.05; 80.40 -> 84.95; 85.08 -> 89.42; 68.79 -> 72.73 | Helped |
| use FIA-USA instead of the reported baseline detectors | EfficientNet-B4 deepfake detector · DF40 face swapping (six methods), Avg. | AUC: 78.1 -> 87.8; 64.4 -> 87.8; 77.8 -> 87.8 | Helped |
| remove MTMS from FIA-USA (ablation) | FIA-USA detector · DiFF FE | frame-level AUC: 71.93 -> 72.73 | Helped |
| add MTMS to FIA-USA (ablation) | FIA-USA detector · CDF-v2 | video-level AUC: 91.99 -> 94.10 | Hurt |
| add MTMS to FIA-USA (ablation) | FIA-USA detector · DFDCP | video-level AUC: 87.06 -> 86.66 | Hurt |
| add SR reconstruction (ablation) | FIA-USA detector · DiFF FS and FE | frame-level AUC: 80.73 -> 89.42; 61.07 -> 72.73 | Helped |
| add AE reconstruction (ablation) | FIA-USA detector · DiFF FS and FE | frame-level AUC: 88.67 -> 89.42; 70.31 -> 72.73 | Mixed |
| add FIA-USA instead of SBI augmentation (ablation) | EfficientNet-B4 deepfake detector · DiFF FS and FE | frame-level AUC: 80.68 -> 89.42; 62.99 -> 72.73 | Helped |
| add PCR (ablation) | FIA-USA detector with RCR · CDF-v2 | video-level AUC: 90.12 -> 94.10 | Helped |
| add PCR (ablation) | FIA-USA detector with RCR · DFDCP | video-level AUC: 86.13 -> 86.66 | Helped |
| add ICR (ablation) | FIA-USA detector with RCR · CDF-v2 | video-level AUC: 90.85 -> 94.10 | Helped |
| add AFFS (ablation) | FIA-USA detector · CDF-v2 | video-level AUC: 94.70 -> 94.10 | No clear effect |
Surprising findings
- Image-level pseudo-fake augmentation reportedly outperforms video-level augmentation methods, and RGB-level artifacts remain discriminative against generative deepfakes despite recent findings of RGB-based method limitations.
- Removing RCR (w/o RCR) dropped the average ablation AUC substantially, while the MTMS component reduced performance on generative forgeries (FS/FE).
Limitations
- Focuses on image-level augmentation; video-level temporal and audio-visual artifacts are not addressed.
- Performance varies across backbones and some forgery edge cases remain weak (e.g., DF40 misclassifications).
- Limited exploration of combining the method with large vision-language models for deepfake detection.
Experimental setup
Datasets: FaceForensics++ c23 DFD DFDC DFDCP CelebDF DiFF DF40 DeepfakeBench
Benchmarks: FaceForensics++ c23 CelebDF-v1 CelebDF-v2 DFD DFDCP DFDC DiFF DF40
Metrics: frame-level AUC video-level AUC AUC reduction ratio
Baselines: Face X-ray SBI SLADD SeeABLE Plug-and-Play LSDA ProDet RECCE UCF LAA-Net
Builds on: SBI Face X-ray LAA-Net FPN
Closest papers by meaning
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.91)
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion (NeurIPS 2024 · similarity 0.91)
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.90)
- D^3: Scaling Up Deepfake Detection by Learning from Discrepancy (CVPR 2025 · similarity 0.89)
- Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning (CVPR 2025 · similarity 0.89)
- FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.