Research map / Detection and segmentation / CNN features
MGQFormer: Mask-Guided Query-Based Transformer for Image Manipulation Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MGQFormer, a query-based transformer for image manipulation localization that uses ground-truth-mask-derived guiding query tokens and a mask-guided loss to speed convergence and improve localization.
Problem
Image manipulation (tampering) localization models train inefficiently because cross-entropy on ground-truth masks ignores spatial location and shape of forged regions, and CNN decoders lack global context.
Approach
A query-based transformer decoder with learnable authentic/forged query tokens, supervised during training by guiding query tokens extracted from noised ground-truth masks; a cosine mask-guided loss pulls the learnable queries toward the guiding queries, and the guidance is dropped at inference.
Key results
- Pre-trained MGQFormer reaches 90.1% average AUC across Columbia, CASIA, NIST16 and IMD20, versus 89.9% for ERMPC.
- Fine-tuned on CASIA, MGQFormer achieves 91.5% AUC and 58.8% F1, versus 90.4% AUC and 58.6% F1 for ERMPC.
- Mask-guided training lifts first-epoch AUC by 12.7% on the synthesized validation split and removing it drops Columbia AUC by 2.8% and CASIA AUC by 3.6%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MGQFormer instead of ERMPC | Pre-trained manipulation localization models · Columbia | AUC (%): 96.8 -> 97.1 | Helped |
| Use MGQFormer instead of ERMPC | Pre-trained manipulation localization models · CASIA | AUC (%): 87.6 -> 88.6 | Helped |
| Use MGQFormer instead of ERMPC | Pre-trained manipulation localization models · NIST16 | AUC (%): 89.5 -> 86.2 | Hurt |
| Use MGQFormer instead of ERMPC | Pre-trained manipulation localization models · IMD20 | AUC (%): 85.6 -> 88.3 | Helped |
| Use MGQFormer instead of ERMPC | Fine-tuned manipulation localization models · CASIA test split | AUC (%): 90.4 -> 91.5 | Helped |
| Use MGQFormer instead of ERMPC | Fine-tuned manipulation localization models · CASIA test split | F1 (%): 58.6 -> 58.8 | Helped |
| Add a noise branch and SCAM (ablation) | MGQFormer · Columbia | AUC (%): 96.0 -> 97.1 | Helped |
| Add a noise branch and SCAM (ablation) | MGQFormer · CASIA | AUC (%): 86.3 -> 88.6 | Helped |
| Add mask-guided training (ablation) | MGQFormer · Columbia | AUC (%): 94.3 -> 97.1 | Helped |
| Add mask-guided training (ablation) | MGQFormer · CASIA | AUC (%): 85.0 -> 88.6 | Helped |
| Add point noise to GT guiding masks (ablation) | MGQFormer mask-guided training · Columbia | AUC (%): 96.2 -> 97.1 | Helped |
| Add point noise to GT guiding masks (ablation) | MGQFormer mask-guided training · CASIA | AUC (%): 87.4 -> 88.6 | Helped |
Surprising findings
- MGQFormer does not achieve the best result on NIST16, which the authors attribute to high-resolution test images exceeding the 384x384 training resolution.
- Under Gaussian blur with k=15, MGQFormer's AUC drops by 19.26 points, indicating sensitivity to heavy blurring.
Limitations
- Performance degrades with test resolution far above the 384x384 training resolution (NIST16).
- Sensitive to strong blur (Columbia, k=15 drops AUC substantially).
Experimental setup
Datasets: PSCC-Net synthesized dataset CASIA
Benchmarks: CASIA Columbia NIST16 IMD20
Metrics: AUC F1
Baselines: ManTraNet SPAN ObjectFormer ERMPC RGB-N PSCC-Net
Builds on: DN-DETR Mask2Former PSCC-Net ViT
Closest papers by meaning
- Towards Modern Image Manipulation Localization: A Large-Scale Dataset and Novel Methods (CVPR 2024 · similarity 0.86)
- FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models (ICLR 2025 · similarity 0.85)
- MUN: Image Forgery Localization Based on M³ Encoder and UN Decoder (AAAI 2025 · similarity 0.85)
- Learning Discriminative Noise Guidance for Image Forgery Detection and Localization (AAAI 2024 · similarity 0.85)
- Can We Model the Artifacts Explicitly? Disentangle Artifacts via Pairwise Edit Relations for Image Manipulation Localization (NeurIPS 2026 · similarity 0.85)
- Training-Free Image Manipulation Localization Using Diffusion Models (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.