Research map / Label-efficient and robust learning / Recognition and long-tail
SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SUMI-IFL, an information-theoretic image forgery localization framework that adds a sufficiency-view constraint on a multi-backbone feature extractor and a minimality-view (conditional entropy bottleneck) constraint on the reasoning network, reporting gains over several IFL baselines in-dataset and cross-dataset.
Problem
Image forgery localization must extract comprehensive forgery clues while suppressing task-unrelated noise (JPEG artifacts, reconstruction artifacts), but existing methods lack theoretical guarantees for feature sufficiency and minimality.
Approach
Derives two information-theoretic losses: a sufficiency loss from a mutual-information chain-rule bound over three attention U-Net backbones (channel, spatial, pixel) fused by a learnable layer, and a minimality loss from the conditional entropy bottleneck, approximated by a mask-guided encoder that maps noised ground-truth masks into feature space.
Key results
- On in-dataset comparisons SUMI-IFL reaches F1 0.9249 / AUC 0.9760 on DEFACTO-12 and F1 0.7995 on SSRGFD, ahead of the next-best methods.
- In cross-dataset tests on NIST16 SUMI-IFL attains F1 0.6178 and AUC 0.7339, with the paper stating F1 exceeds the second-best method by 9.7%.
- Ablation on DEFACTO-12 and SSRGFD shows removing the sufficiency loss drops F1 by 9.8% on DEFACTO-12 and removing the minimality loss by 5.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · DEFACTO-12 | F1: 0.8395 -> 0.9249 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · DEFACTO-12 | AUC: 0.9504 -> 0.9760 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · SSRGFD | F1: 0.6734 -> 0.7995 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · SSRGFD | AUC: 0.8866 -> 0.9493 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · CASIAv2 | F1: 0.7347 -> 0.7604 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · CASIAv2 | AUC: 0.8941 -> 0.8984 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · CIMD | F1: 0.1707 -> 0.1738 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · CIMD | AUC: 0.5598 -> 0.5513 | Hurt |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · NIST16 | F1: 0.5519 -> 0.6178 | Helped |
| use SUMI-IFL instead of the strongest listed baseline | SUMI-IFL · NIST16 | AUC: 0.7017 -> 0.7339 | Helped |
| add the sufficiency-view constraint (L_SU) (ablation) | SUMI-IFL · DEFACTO-12 and SSRGFD | F1: 0.8335 -> 0.9249; 0.6783 -> 0.7995 | Helped |
| add the minimality-view constraint (L_MI) (ablation) | SUMI-IFL · DEFACTO-12 and SSRGFD | F1: 0.8774 -> 0.9249; 0.6784 -> 0.7995 | Helped |
Surprising findings
- The HiFi-Net and PSCC-Net baselines score very low F1 (e.g., HiFi-Net 0.2235 on DEFACTO-12, 0.0977 on SSRGFD), suggesting weak reproduction or domain mismatch.
- On CIMD SUMI-IFL's AUC (0.5513) is below EITL-Net's (0.5598), so it does not win every metric.
Limitations
- Cross-dataset AUC on CIMD is weak (about 0.55) and the paper notes all methods degrade on small-region tampering.
- Theoretical derivations rely on variational approximations and the sufficiency bound is loose (the Q1 term is dropped), so guarantees are approximate.
- Hardware is described as an NVIDIA GTX GeForce A100 with no GPU count, and no code is released.
Experimental setup
Datasets: DEFACTO-12 SSRGFD CASIAv2 Spliced COCO COCO 2017
Benchmarks: DEFACTO-12 SSRGFD CASIAv2 CIMD NIST16
Metrics: F1 AUC
Baselines: MMFusion EITL-Net HiFi-Net WSCL IF-OSN MVSS-Net PSCC-Net
Builds on: information bottleneck conditional entropy bottleneck U-Net DN-DETR
Closest papers by meaning
- Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis (NeurIPS 2025 · similarity 0.88)
- FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models (ICLR 2025 · similarity 0.88)
- M²RL-Net: Multi-View and Multi-Level Relation Learning Network for Weakly-Supervised Image Forgery Detection (AAAI 2025 · similarity 0.87)
- Learning Discriminative Noise Guidance for Image Forgery Detection and Localization (AAAI 2024 · similarity 0.87)
- A Lottery Ticket Hypothesis Approach with Sparse Fine-tuning and MAE for Image Forgery Detection and Localization (AAAI 2025 · similarity 0.87)
- OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.