Research map / Image generation / Autoregressive and latent generation
Training-Free Image Manipulation Localization Using Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free image manipulation localization method that adaptively picks diffusion timesteps and contrasts conditional (similarity + self-attention guided) and unconditional DDPM reconstructions to produce localization maps.
Problem
Image manipulation localization typically requires extensive supervised training on labeled datasets and generalizes poorly to unseen manipulation types.
Approach
Uses a pretrained ImageNet diffusion model with no training: timestep T is chosen adaptively via SSIM on SRM high-pass-filtered forward samples, and the error map is the squared difference between condition-free conditional (similarity- and self-attention-guided) and unconditional backward reconstructions, aggregated from the SSIM-minimum timestep.
Key results
- On five standard IML datasets the training-free method reaches an average AUC of 0.627 and AP of 0.212, beating all unsupervised baselines (best prior NOI1 0.579 AUC / 0.174 AP).
- On the novel MagicBrush dataset it achieves AUC 0.543 and AP 0.206, outperforming fully-supervised Mantra-Net, PSCC-Net, CAT-Net, Hifi-Net and weakly-supervised CR-CNN and WSCL, but trailing MVSS-Net (0.578 AUC / 0.270 AP).
- Ablation on CASIAv1 shows the combined similarity and self-attention guidance gives AUC 0.587 / AP 0.162, better than either alone or the fixed timesteps tested.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed training-free diffusion method instead of NOI1 | IML · CASIAv1 | AUC: 0.586 -> 0.587 | Helped |
| use the proposed training-free diffusion method instead of CFA1 | IML · Columbia | AUC: 0.641 -> 0.682 | Helped |
| use the proposed training-free diffusion method instead of NOI2 | IML · Coverage | AUC: 0.598 -> 0.622 | Helped |
| use the proposed training-free diffusion method instead of BLK | IML · NIST16 | AUC: 0.583 -> 0.556 | Hurt |
| use the proposed training-free diffusion method instead of MVSS-Net | IML · MagicBrush | AUC: 0.578 -> 0.543 | Hurt |
| add similarity guidance and self-attention guidance to unconditional reconstruction (ablation) | proposed diffusion IML method · CASIAv1 | AUC: 0.544 -> 0.587 | Helped |
| add self-attention guidance to similarity-guided reconstruction (ablation) | proposed diffusion IML method · CASIAv1 | AUC: 0.514 -> 0.587 | Helped |
| add similarity guidance to self-attention-guided reconstruction (ablation) | proposed diffusion IML method · CASIAv1 | AUC: 0.573 -> 0.587 | Helped |
| replace adaptive timestep selection with a fixed diffusion timestep (ablation) | proposed diffusion IML method · CASIAv1 | AUC: 0.587 vs. 0.532, 0.577, 0.558, 0.467 | Hurt |
Surprising findings
- Similarity guidance alone reduced performance below the unconditional-only baseline on CASIAv1 (AUC 0.514 vs 0.544), and the method does not beat BLK on NIST16 AUC because NIST16 is JPEG-compressed.
Limitations
- Localization boundaries of tampered regions are imprecise because no pixel-level supervision is used.
- Does not outperform BLK on NIST16 AUC, likely due to JPEG compression.
- Trails MVSS-Net on MagicBrush AP (0.206 vs 0.270), which uses naive manipulation augmentation.
Experimental setup
Datasets: CASIAv1 Columbia Coverage NIST16 CIMD MagicBrush
Benchmarks: CASIAv1 Columbia Coverage NIST16 CIMD MagicBrush
Metrics: AUC AP
Baselines: NOI1 CFA1 MCA NoisePrint IVC CFA2 NOI2 NOI4 BLK Mantra-Net PSCC-Net CAT-Net
Builds on: DDPM classifier-free guidance diffusion purification self-attention guidance Che et al. 2024 error map aggregation
Closest papers by meaning
- Less Supervision, Better Generalization: Weakly Supervised Fake Region Localization in Diffusion-Edited Images (NeurIPS 2026 · similarity 0.89)
- Towards Modern Image Manipulation Localization: A Large-Scale Dataset and Novel Methods (CVPR 2024 · similarity 0.88)
- DiffForensics: Leveraging Diffusion Prior to Image Forgery Detection and Localization (CVPR 2024 · similarity 0.87)
- Can We Model the Artifacts Explicitly? Disentangle Artifacts via Pairwise Edit Relations for Image Manipulation Localization (NeurIPS 2026 · similarity 0.87)
- Mesoscopic Insights: Orchestrating Multi-Scale & Hybrid Architecture for Image Manipulation Localization (AAAI 2025 · similarity 0.87)
- Towards Reliable Identification of Diffusion-based Image Manipulations (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.