Research map / Vision-language and multimodal / Multimodal LLMs
Omni-IML: Towards Unified Interpretable Image Manipulation Localization
Chenfan Qu, Yiwu Zhong, Fengjun Guo, Lianwen Jin
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Omni-IML, a generalist model that unifies image manipulation localization across multiple IML tasks to avoid the performance degradation seen when jointly training task-specific IML methods.
Problem
Existing image manipulation localization (IML) methods rely on task-specific designs and degrade substantially when jointly trained across multiple IML tasks, limiting real-world use.
Approach
A single generalist IML model designed to generalize across diverse manipulation localization tasks, positioned as the first such unified model; the abstract is truncated so the specific mechanism is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| joint training existing IML methods on all four tasks versus a single task (ablation) | TIFDM (document IML) · Document IML | pixel IoU: .498 -> .428 | Hurt |
| joint training Omni-IML on all four tasks versus document only (ablation) | Omni-IML · Document IML | pixel IoU: .774 -> .766 | No clear effect |
| add Modal Gate (versus fused frequency-vision features without gate) (ablation) | Omni-IML · All four tasks (average) | pixel IoU: .629 -> .728 | Improved |
| add Modal Gate (versus pure vision modality without gate) (ablation) | Omni-IML · All four tasks (average) | pixel IoU: .659 -> .728 | Improved |
| add Dynamic Weight Decoder (ablation) | Omni-IML · All four tasks (average) | pixel IoU: .608 -> .728 | Improved |
| make decoder filter weights input-adaptive instead of fixed (ablation) | Dynamic Weight Decoder · All four tasks (average) | pixel IoU: .688 -> .728 | Improved |
| add Anomaly Enhancement with box supervision (ablation) | Omni-IML · All four tasks (average) | pixel IoU: .682 -> .728 | Improved |
| add all three proposed modules to the baseline (ablation) | Baseline · All four tasks (average) | pixel IoU: .575 -> .728 | Improved |
| feed a visual reference prompt (mask-highlighted image) to the MLLM | Qwen2.5-VL 7B · Omni-273k documents: tampered text recognition | OCR accuracy: .312 -> .653 | Improved |
| feed a visual reference prompt to the MLLM | Qwen2.5-VL 7B · Omni-273k interpretation (average) | average score: .512 -> .603 | Improved |
| visual reference prompt on face and natural image absolute position/description | InternVL3 2B · Omni-273k natural image: tampered object recognition | accuracy: .778 -> .770 (face Obj. Rec. .757 -> .770) | No clear effect |
| Omni-IML versus best prior on DocTamper SCD and CASIA1 | Omni-IML · CASIA1 and DT-SCD | pixel IoU: CASIA1 .800 vs .810; DT-SCD .676 vs .691 | Mixed |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- FakeShield 13B — FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models (ICLR 2025) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Surprising findings
- Joint training of existing task-specific IML methods on multiple IML tasks causes significant performance degradation.
Closest papers by meaning
- Training-Free Image Manipulation Localization Using Diffusion Models (AAAI 2025 · similarity 0.89)
- Can We Model the Artifacts Explicitly? Disentangle Artifacts via Pairwise Edit Relations for Image Manipulation Localization (NeurIPS 2026 · similarity 0.88)
- Can We Get Rid of Handcrafted Feature Extractors? SparseViT: Nonsemantics-Centered, Parameter-Efficient Image Manipulation Localization Through Spare-Coding Transformer (AAAI 2025 · similarity 0.87)
- Mesoscopic Insights: Orchestrating Multi-Scale & Hybrid Architecture for Image Manipulation Localization (AAAI 2025 · similarity 0.87)
- OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL (ICML 2026 · similarity 0.86)
- UnionFormer: Unified-Learning Transformer with Multi-View Representation for Image Manipulation Detection and Localization (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
