Research map / Vision-language and multimodal / Multimodal LLMs

The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contexts

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds MDSM, a 441k-sample dataset of face-swapped/face-attribute-edited news images paired with MLLM-fabricated, semantically aligned text, and proposes AMD, a Florence-2-based text-output framework with artifact token pre-perception and manipulation-oriented reasoning for detecting, classifying, and grounding such manipulations.

Problem

Existing multimodal manipulation detection and grounding benchmarks (e.g., DGM^4) use rule-based text edits and independently manipulated image/text, producing semantically misaligned pairs that are easy to detect; they miss deception by MLLM-generated coherent narratives.

Approach

Generate semantically coherent disinformation by feeding the manipulated image and face-edit metadata to an MLLM to write aligned fake text, and tackle detection with a unified sequence-to-sequence MLLM (Florence-2) that learns a learnable artifact token via pre-perception, auxiliary losses, and a token redundancy penalty, outputting detection, type, and bbox as text.

Key results

• On MDSM cross-domain (trained on NYT/Guardian), AMD achieves average 88.18 ACC, 60.25 mAP, 61.02 mIoU in the Guardian-trained setting.
• On DGM^4, AMD reaches 74.47 ACC, 52.91 mAP, 80.01 P_tok, 51.87 mIoU averaged across cross-domain test sets, outperforming HAMMER series on all metrics.
• AMD has 276.95M parameters, smaller than FKA-Owl (6,771.98M total params).

Surprising findings

• General-purpose zero-shot MLLMs (e.g., Gemini-2.0, Qwen3-VL-235B) achieve mIoU below 3 on MDSM grounding, failing to localize manipulated faces.
• Text-fabrication (TF) manipulations are harder to detect than image-based manipulations, with cross-domain AP 57.53 for TF versus 71.37 for FA.

Limitations

• Dataset release is restricted under a data usage agreement and generation pipeline and prompts are withheld for dual-use safety, limiting reproducibility.
• Among comparable-sized models, AMD is slower than HAMMER in throughput (13.38 vs 61.28 pairs/s).

Experimental setup

Datasets: MDSM DGM^4 GoodNews VisualNews N24News Celebrity Head-shot Dataset

Benchmarks: MDSM DGM^4

Metrics: ACC mAP mIoU Precision of fake token grounding

Baselines: HAMMER HAMMER++ FKA-Owl ViLT LLaVA-v1.6-13B DeepSeek-VL2-27B Yi-VL-34B Qwen2.5-VL-72B Qwen3-VL-235B GPT-4o Gemini-2.0

Builds on: Florence-2 HAMMER DGM^4 FKA-Owl

Closest papers by meaning

• Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models (ICML 2026 · similarity 0.88)
• [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning (NeurIPS 2026 · similarity 0.87)
• ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection (NeurIPS 2025 · similarity 0.87)
• CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection (ICML 2026 · similarity 0.87)
• When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm (CVPR 2026 · similarity 0.86)
• Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models (ICLR 2026 · similarity 0.86)