Research map / Image generation / Autoregressive and latent generation
DiffForensics: Leveraging Diffusion Prior to Image Forgery Detection and Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffForensics, a two-stage method that pretrains a SegFormer-encoder (frozen, ADE20K) + UNet-style decoder with Simplex-noise DDPM denoising, then fine-tunes for image forgery detection and localization with an Edge Cue Enhancement Module.
Problem
Image forgery detection and localization (IFDL) suffers from poor generalization and robustness of current encoder-decoder detectors, which rely on ImageNet/macroscopic features.
Approach
Combine a frozen segmentation-pretrained encoder (macroscopic features) with a decoder trained via self-supervised denoising diffusion using Simplex noise (microscopic/mesoscopic traces), then fine-tune with multi-task detection, localization and multi-scale edge supervision via an Edge Cue Enhancement Module.
Key results
- Average pixel-level localization F1 of 0.466 and AUC of 0.862 across eight datasets, the best among compared methods.
- Average image-level detection AUC of 0.916 and EER of 0.131 across four datasets, with the best AUC and lowest EER.
- On IMD2020 and AutoSplicing, Simplex-noise DDPM pretraining with frozen ADE20K encoder and DDPM decoder reaches average F1 0.635 and AUC 0.886 in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Simplex instead of Gaussian noise for DDPM pretraining with the ADE20K encoder (ablation) | DiffForensics · IMD2020 and AutoSplicing | pixel-level F1: .470 -> .511; .346 -> .507 | Helped |
| use Simplex instead of Gaussian noise for DDPM pretraining with the ADE20K encoder (ablation) | DiffForensics · IMD2020 and AutoSplicing | pixel-level AUC: .904 -> .911; .918 -> .940 | Helped |
| use the ADE20K encoder and DDPM-train only the decoder instead of DDPM-training both (ablation) | DiffForensics · IMD2020 and AutoSplicing | pixel-level F1: .380 -> .511; .304 -> .507 | Helped |
| add DDPM pretraining with Simplex noise to the ADE20K encoder baseline (ablation) | DiffForensics · IMD2020 and AutoSplicing | pixel-level F1: .416 -> .511; .297 -> .507 | Helped |
| add multi-scale, weighted ECEM edge supervision (ablation) | DiffForensics · IMD2020 and AutoSplicing | localization F1: .316 -> .509 | Helped |
| add ECEM edge supervision at all decoder scales instead of only the last scale (ablation) | DiffForensics · IMD2020 and AutoSplicing | localization F1: .409 -> .509 | Helped |
| use weighted rather than unweighted segmentation BCE (ablation) | DiffForensics · IMD2020 and AutoSplicing | localization F1: .387 -> .509 | Helped |
| use weighted rather than unweighted classification BCE (ablation) | DiffForensics · IMD2020 and AutoSplicing | localization F1: .452 -> .509 | Helped |
| use fixed time embedding t=5 instead of t=0 (ablation) | DiffForensics · IMD2020 and AutoSplicing | average F1: .513 -> .635 | Helped |
| use fixed time embedding t=5 instead of t=10 (ablation) | DiffForensics · IMD2020 and AutoSplicing | average F1: .526 -> .635 | Helped |
| use fixed time embedding t=5 instead of t=500 (ablation) | DiffForensics · IMD2020 and AutoSplicing | average F1: .445 -> .635 | Helped |
| use DiffForensics rather than HiFi-Net for localization | Image forgery localization · CASIA1.0+, Columbia, NIST16, IMD2020, DSO-1, and Korus | pixel-level F1: .092 -> .517; .382 -> .912; .172 -> .415; .178 -> .511; .304 -> .485; .088 -> .257 | Helped |
Surprising findings
- Gaussian-noise DDPM pretraining with a DDPM-pretrained encoder performed worse than no DDPM pretraining on some settings, suggesting encoder diffusion training may cause catastrophic forgetting of macroscopic features.
Limitations
- Detection performance on some datasets (e.g., ACC on IMD2020) is not best due to class imbalance and threshold sensitivity.
- Baselines were retrained under the authors' settings, so reported numbers depend on those reproductions.
Experimental setup
Datasets: CASIAv2.0 Fantastic-Reality CASIAv1+ Columbia NIST16 IMD2020 DSO-1 Korus AutoSplice OpenForensics
Benchmarks: CASIAv1+ Columbia NIST16 IMD2020 DSO-1 Korus AutoSplice OpenForensics
Metrics: pixel-level F1 AUC image-level ACC EER
Baselines: H-LSTM ManTra-Net HP-FCN GSR-Net SPAN MVSS-Net CAT-Net SATL-Net PSCC-Net HiFi-Net
Builds on: DDPM SegFormer U-Net MesoNet Simplex noise
Closest papers by meaning
- Learning Discriminative Noise Guidance for Image Forgery Detection and Localization (AAAI 2024 · similarity 0.88)
- Training-Free Image Manipulation Localization Using Diffusion Models (AAAI 2025 · similarity 0.87)
- A Lottery Ticket Hypothesis Approach with Sparse Fine-tuning and MAE for Image Forgery Detection and Localization (AAAI 2025 · similarity 0.86)
- Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization (AAAI 2025 · similarity 0.86)
- On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection (NeurIPS 2024 · similarity 0.86)
- ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localization (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.