Research map / Detection and segmentation / CNN features
Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neighboring Pixel Relationships (NPR), a local pixel-difference artifact representation from up-sampling layers in GAN/diffusion generators, and trains a lightweight CNN on ProGAN images to detect GAN and diffusion images across 28 generators.
Problem
Generalizable deepfake detection: detectors trained on one generator source often fail on unseen GAN and diffusion sources, and prior work mostly analyzes up-sampling artifacts in the frequency domain.
Approach
Captures local up-sampling artifacts in image space by computing differences between pixels within each l×l grid (e.g., 2×2), which reflect correlations introduced by up-sampling and translation-invariant convolutions, and uses these relative relationships as the artifact representation for a small CNN classifier.
Key results
- NPR reaches 92.5% mean accuracy on the ForenSynths test set (vs. 89.1% for Ojha and 86.1% for LGrad) when trained on ProGAN.
- Across 28 generation models, NPR achieves 93.3% mean accuracy, outperforming Ojha and LGrad by 13.5% and 12.8%.
- On the DiffusionForensics benchmark NPR reaches 95.3% mean accuracy and 99.8% AP, versus 88.2% for LGrad.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the detector on NPR rather than the strongest listed baseline | Deepfake detection trained on ProGAN · 28 generation models, 38 sub-testsets | Mean accuracy: 80.5 -> 93.3 | Helped |
| Use w2 rather than w1 as the 2×2 NPR reference pixel (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 87.5 | Hurt |
| Use w3 rather than w1 as the 2×2 NPR reference pixel (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 91.0 | Hurt |
| Use w4 rather than w1 as the 2×2 NPR reference pixel (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 91.4 | Hurt |
| Use the 2×2 patch mean rather than w1 as the NPR reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 88.3 | Hurt |
| Use the 2×2 patch maximum rather than w1 as the NPR reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 90.9 | Hurt |
| Increase NPR patch size from 2×2 to 3×3, using w1 (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 89.4 | Hurt |
| Increase NPR patch size to 3×3 and use w2 as reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 90.1 | Hurt |
| Increase NPR patch size to 3×3 and use w3 as reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 89.0 | Hurt |
| Increase NPR patch size to 3×3 and use w4 as reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 89.6 | Hurt |
| Increase NPR patch size to 3×3 and use the patch mean as reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 87.1 | Hurt |
| Increase NPR patch size to 3×3 and use the patch maximum as reference (ablation) | NPR detector · ForenSynths test set | Acc.: 92.5 -> 90.5 | Hurt |
Surprising findings
- A detector trained only on ProGAN images with a 1.44M-parameter network generalizes to diffusion models, including Midjourney and DALLE, despite the diffusion models using different up-sampling pipelines.
- Performance on 1000-step diffusion images is much lower (80.1% mean accuracy), indicating the method is weaker on higher-fidelity diffusion outputs.
Limitations
- Performance drops on diffusion models with 1000 sampling steps (80.1% mean accuracy) and on some GANs such as STGAN and SNGAN in the self-synthesis set.
- Relies on the assumption that up-sampling layers leave local pixel correlations; generators without such layers are not covered.
Experimental setup
Datasets: ForenSynths (ProGAN training set, LSUN real) ForenSynths test set GANGen 9 GANs DIRE DiffusionForensics Ojha diffusion dataset Diffusion1kStep (self-collected Midjourney, DALLE, 1000-step diffusions)
Benchmarks: ForenSynths test set Self-Synthesis 9 GANs DiffusionForensics Ojha diffusion test set Self-Synthesis Diffusion dataset
Metrics: accuracy average precision
Baselines: CNNDetection Frank Durall Patchfor F3Net SelfBland GANDetection BiHPF FrePGAN LGrad Ojha
Builds on: CNNDetection FreGAN LGrad Ojha et al. universal fake detection BiHPF
Closest papers by meaning
- D^3: Scaling Up Deepfake Detection by Learning from Discrepancy (CVPR 2025 · similarity 0.88)
- Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection (AAAI 2025 · similarity 0.85)
- Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Domain Learning (AAAI 2024 · similarity 0.84)
- From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes (NeurIPS 2025 · similarity 0.83)
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion (NeurIPS 2024 · similarity 0.83)
- Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.