Research map / Detection and segmentation / CNN features
Critical Forgetting-Based Multi-Scale Disentanglement for Deepfake Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a disentangled representation learning framework for deepfake detection that decouples encoder feature maps (not raw images) at multiple scales, plus a gradient-guided "critical forgetting" mask on attribute features, evaluated by cross-dataset AUC/EER on FF++, CDF, DFDC and WDF.
Problem
Cross-reconstruction disentanglement methods for face forgery detection use information-dense images as decoupling targets, assume reconstruction-irrelevant attributes are forgery-relevant, and produce attribute features that overfit training-set artifacts, which hurts generalization.
Approach
Use single-image encoder feature maps as decoupling targets, decouple at multiple scales with feature-level reconstruction and a Pearson decorrelation loss, and apply a critical forgetting mechanism that masks the top gradient-ranked attribute channels before classification while concatenating original and masked features as augmentation.
Key results
- Trained on FF++ (c40), reaches 76.30% AUC on CDF vs 74.42% for the prior best PFG, and 71.95% average AUC over CDF, WDF and DFDC.
- On FF++ (c23) intra-dataset, achieves 98.7% AUC, about 0.9 points below UCF (99.6%), while reaching 84.9% on CDF (best reported).
- Ablation on Xception backbone: adding multi-scale decoupling and critical forgetting raises CDF AUC from 70.80 to 75.91 (with contrastive learning reaching 76.30 on EfficientNet).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add feature decoupling instead of decoupling images (ablation) | Xception detector · CDF (FF++ c40 cross-dataset) | AUC: 70.80 -> 72.45 | Helped |
| Add multi-scale forgery representation decoupling (ablation) | Xception detector with feature decoupling · CDF (FF++ c40 cross-dataset) | AUC: 72.45 -> 74.94 | Helped |
| Add critical forgetting mechanism (ablation) | Xception detector with multi-scale decoupling · CDF (FF++ c40 cross-dataset) | AUC: 74.94 -> 75.91 | Helped |
| Add contrastive learning (ablation) | Xception detector with critical forgetting · CDF (FF++ c40 cross-dataset) | AUC: 75.91 -> 76.30 | Helped |
| Apply the proposed framework to Xception instead of using the backbone alone (ablation) | Xception · CDF (FF++ c40 cross-dataset) | AUC: 69.21 -> 76.64 | Helped |
| Apply the proposed framework to Xception instead of using the backbone alone (ablation) | Xception · WDF (FF++ c40 cross-dataset) | AUC: 66.88 -> 69.02 | Helped |
| Apply the proposed framework to Xception instead of using the backbone alone (ablation) | Xception · DFDC (FF++ c40 cross-dataset) | AUC: 64.52 -> 68.99 | Helped |
| Use the proposed method rather than UCF | Deepfake detector · FF++ (c23) | AUC: 99.6 vs. 98.7 | Hurt |
| Use the proposed method rather than DisGRL | Deepfake detector · DFDC (FF++ c40 cross-dataset) | AUC: 70.89 vs. 70.27 | Hurt |
| Use the proposed method rather than SLADD | Deepfake detector · FS c40 (trained on NT c23) | AUC: 56.8 vs. 55.6 | Hurt |
| Use the proposed method rather than Face X-ray | Deepfake detector · FS c23 (trained on NT c23) | AUC: 77.9 vs. 71.1 | Hurt |
Surprising findings
- Attribute features from cross-reconstruction decoupling do not reliably outperform the plain backbone, even in cross-dataset settings (Figure 1).
- The method is slightly worse than UCF in the intra-dataset FF++ (c23) setting, which the authors attribute to forcing generalized rather than specific artifacts.
Limitations
- Forgetting ratio sigma is chosen manually based on experience.
- Slightly lower intra-dataset AUC than UCF on FF++ (c23).
- Mixed results on some cross-manipulation settings (e.g., FS at c40 below SLADD).
Experimental setup
Datasets: FaceForensics++ (c40) FaceForensics++ (c23) NeuralTextures (c23)
Benchmarks: FaceForensics++ Celeb-DF v2 DFDC WildDeepfake DeepFake cross-manipulation (DF, FS)
Metrics: AUC EER
Baselines: Xception F3Net Add-Net RFM MAT RECCE UCF DisGRL PFG Face X-ray LTW PCL
Builds on: Representative Forgery Mining Deep InfoMax UCF DisGRL
Closest papers by meaning
- Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection (AAAI 2024 · similarity 0.89)
- DF40: Toward Next-Generation Deepfake Detection (NeurIPS 2024 · similarity 0.89)
- Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector (CVPR 2025 · similarity 0.88)
- FakeDiffer: Distributional Disparity Learning on Differentiated Reconstruction for Face Forgery Detection (AAAI 2025 · similarity 0.87)
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.87)
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.