Research map / Image generation / Autoregressive and latent generation
Towards Modern Image Manipulation Localization: A Large-Scale Dataset and Novel Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CAAA, a category-aware auto-annotation paradigm with a QES quality metric, uses it to build the 123,150-image MIML manipulation-localization dataset, and introduces the APSC-Net model.
Problem
Image manipulation localization is limited by the scarcity of high-quality, manually forged training data, since pixel-level annotation is costly and synthetic data generalizes poorly.
Approach
Split image pairs into Shared Probe Group (SPG) and Shared Donor Group (SDG) with a self-supervised classifier, handle each with a specialized constrained-localization model (difference-aware segmentation for SPG, semantic-aligned correlation matching for SDG), auto-annotate web-collected forged images, and filter labels with a confidence/sharpness score (QES).
Key results
- Training with MIML improves PSCC-Net and CAT-Netv2 on CASIAv1, NIST16, Coverage and IMD20; e.g., IMD20 IoU rises from .125 to .470 for PSCC-Net and from .157 to .547 for CAT-Netv2 (per Table 2 pairing).
- APSC-Net with MIML reaches IoU .810 / F1 .848 on CASIAv1 and IoU .525 / F1 .590 on NIST16 (Table 7).
- On IMD20 SDG, the proposed correlation matching reaches IoU .903 with QES filtering versus .573 for DMAC (Tables 4-5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use APSC-Net with MIML training data rather than without MIML (ablation) | APSC-Net · CASIAv1 | IoU: .799 -> .810 | Helped |
| Use APSC-Net with MIML training data rather than without MIML (ablation) | APSC-Net · NIST16 | IoU: .398 -> .525 | Helped |
| Use APSC-Net with MIML training data rather than without MIML (ablation) | APSC-Net · Coverage | IoU: .490 -> .498 | Helped |
| Use APSC-Net with MIML training data rather than without MIML (ablation) | APSC-Net · Columbia | IoU: .956 -> .962 | Helped |
| Add the difference map to the semantic segmentation model input (ablation) | DASS on IMD20 SPG · IMD20 SPG | IoU: .718 -> .834 | Helped |
| Add the image-pair input alongside the difference map (ablation) | DASS on IMD20 SPG · IMD20 SPG | IoU: .741 -> .834 | Helped |
| Add top-down fusion (ablation) | SACM on IMD20 SDG · IMD20 SDG | IoU: .590 -> .702 | Helped |
| Add cross-level matching (ablation) | SACM on IMD20 SDG · IMD20 SDG | IoU: .608 -> .702 | Helped |
| Filter predictions using QES > 0.5 (ablation) | SACM on IMD20 SDG · IMD20 SDG | IoU: .702 -> .903 | Helped |
| Increase the QES threshold from 0.0 to 0.5 (ablation) | SACM predictions on IMD20 SDG · IMD20 SDG | IoU: .702 -> .903 | Helped |
| Replace MIML with augmented IMDP1 for training (ablation) | APSC-Net · CASIAv1 | IoU: .810 -> .790 | Hurt |
Limitations
- Annotations are produced by models rather than human labels, so label noise remains; QES filtering only partly addresses this.
- Some IMD20 ground truth contains errors, complicating evaluation of the annotator.
Experimental setup
Datasets: CASIAv2 IMD20 COCO MIML
Benchmarks: CASIAv1 NIST16 Coverage IMD20 Columbia
Metrics: IoU F1
Baselines: DMVN DMAC PSCC-Net CAT-Netv2 ManTraNet RRU-Net MVSS-Net IF-OSN EVP TruFor
Builds on: DMVN DMAC CAT-Net PSCC-Net ConvNeXt
Closest papers by meaning
- Training-Free Image Manipulation Localization Using Diffusion Models (AAAI 2025 · similarity 0.88)
- Can We Model the Artifacts Explicitly? Disentangle Artifacts via Pairwise Edit Relations for Image Manipulation Localization (NeurIPS 2026 · similarity 0.87)
- MGQFormer: Mask-Guided Query-Based Transformer for Image Manipulation Localization (AAAI 2024 · similarity 0.86)
- FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models (ICLR 2025 · similarity 0.86)
- GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization (AAAI 2025 · similarity 0.86)
- UnionFormer: Unified-Learning Transformer with Multi-View Representation for Image Manipulation Detection and Localization (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.