Research map / Detection and segmentation / CNN features
MUN: Image Forgery Localization Based on M³ Encoder and UN Decoder
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MUN, an image forgery localization network combining a dual-stream RGB and Noiseprint++ ConvNeXt V2 encoder with a multi-scale max-pooling query fusion (M3 encoder), a two-branch U/N decoder, an IoU-recalibrated dynamic cross-entropy loss, and a distribution-shifting augmentation (DNA).
Problem
Localizing multi-type image forgeries (splicing, copy-move, removal, and AI-generated inpainting) with good discrimination and generalization across datasets.
Approach
Fuse RGB and Noiseprint++ hierarchical features via a multi-scale max-pooling query (RGB pooled features query noise features by matrix multiplication), decode with parallel bottom-up and top-down branches, weight loss by batch IoU, and augment training images by shifting RGB means toward ImageNet statistics.
Key results
- MUN* (with DNA) reaches AUC 0.897 on IMD2020 and 0.843 on Wild, versus the prior best of 0.883 (MGQFormer) on IMD2020 in Table 6.
- Backbone study on the Synthesized Dataset: ConvNeXt V2-B achieves F1 0.9519, above Swin-B (0.9450) and ResNet-152 (0.9326), with fewer FLOPS.
- Adding the Noiseprint++ branch with resize-after-NPP ordering raises F1 from 0.9525 to 0.9533 on the Synthesized Dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IoUDCE loss instead of cross-entropy loss (ablation) | MUN RGB-only model · Synthesized Dataset test | F1: 0.9519 -> 0.9523 | Helped |
| Add the UN decoder (ablation) | MUN with RGB and IoUDCE · Synthesized Dataset test | F1: 0.9523 -> 0.9525 | Helped |
| Add the Noiseprint++ branch (ablation) | MUN with RGB, IoUDCE, and UN decoder · Synthesized Dataset test | F1: 0.9525 -> 0.9533 | Helped |
| Apply Noiseprint++ before resizing rather than after resizing (ablation) | MUN · Synthesized Dataset test | F1: 0.9521 -> 0.9533 | Helped |
| Use max-pooling kernels 1, 5, 11, and 15 in MMQ rather than no max-pooling streams (ablation) | MUN · Synthesized Dataset test | F1: 0.9533 -> 0.9543 | Helped |
| Add DNA augmentation (ablation) | MUN · CASIA v1.0 | AUC: 0.967 -> 0.962 | Hurt |
| Add DNA augmentation (ablation) | MUN · IMD2020 | AUC: 0.885 -> 0.897 | Helped |
| Add DNA augmentation (ablation) | MUN · CocoGlide | AUC: 0.811 -> 0.815 | Helped |
| Add DNA augmentation (ablation) | MUN · Wild | AUC: 0.805 -> 0.843 | Helped |
| Use ConvNeXt V2-B instead of ResNet-152 as backbone (ablation) | MUN · Synthesized Dataset test | F1: 0.9326 -> 0.9519 | Helped |
Surprising findings
- Applying Noiseprint++ after resizing hurts performance (F1 0.9521 vs 0.9525 for resize only), so NPP must be applied before resizing.
- DNA slightly reduces performance on CASIA v1.0 because the augmentation moves the training distribution away from CASIA v1.0's distribution.
Limitations
- Noiseprint++ GPU memory cost forces patch-based processing on ultra-high-resolution images (NIST16), which lowers the score there.
- Gains over the ablation components are small (e.g., +0.0008 F1 from the UN decoder)
- Robustness to JPEG q=50 and strong blur is weaker than MVSS-Net in some settings
Experimental setup
Datasets: Synthesized Dataset (Liu et al. 2023) ImageNet
Benchmarks: NIST16 CASIA v1.0 IMD2020 CocoGlide Wild Synthesized Dataset
Metrics: F1 IoU Accuracy AUC
Baselines: RGB-N ManTra-Net SPAN MVSS-Net PSCC-Net ObjectFormer TANet TBFormer HiFi TruFor CSR-Net NRL-Net
Builds on: ConvNeXt V2 Noiseprint++ MMSegmentation TBFormer synthesized dataset
Closest papers by meaning
- Learning Discriminative Noise Guidance for Image Forgery Detection and Localization (AAAI 2024 · similarity 0.88)
- A Lottery Ticket Hypothesis Approach with Sparse Fine-tuning and MAE for Image Forgery Detection and Localization (AAAI 2025 · similarity 0.85)
- Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization (AAAI 2025 · similarity 0.85)
- MGQFormer: Mask-Guided Query-Based Transformer for Image Manipulation Localization (AAAI 2024 · similarity 0.85)
- SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints (AAAI 2025 · similarity 0.84)
- FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.