Research map / Vision-language and multimodal / Multimodal LLMs

Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DeceptionDecoded, a 12,000-sample intent-guided synthetic benchmark for detecting misleading creator intent in multimodal news, evaluates 14 VLMs, and shows fine-tuning on it improves transfer to general MMD benchmarks.

Problem

Existing multimodal misinformation benchmarks rely on heuristic mismatches and reader-perceived intent, so VLMs are not evaluated on detecting deliberately embedded misleading creator intent in trustworthy news contexts.

Approach

An intent-guided simulation pipeline in which GPT-4o plays malicious or trustworthy creators (defined by desired influence and execution plan) to generate subtle and significant misleading image or caption variants anchored to VisualNews reference articles, plus three intent-centric tasks.

Key results

• Best VLM (Gemini-2.5-Pro, consistency-oriented) reaches 85.7% average accuracy on text misleading intent detection and 69.2% on image, while small open models such as LLaVA-v1.6-7B collapse to 33.3% (predict all non-misleading).
• Fine-tuning LLaVA-v1.6-7B on 6k DeceptionDecoded samples raises macro-F1 on MMFakeBench from 44.41 to 52.37, Fakeddit from 34.63 to 39.43, and FakeNewsNet from 43.73 to 65.22.
• Fine-tuned Qwen2.5-VL-7B improves MMFakeBench macro-F1 from 27.96 to 58.66 and FakeNewsNet from 44.87 to 68.59.

Surprising findings

• Authenticity hints in prompts (e.g., 'sourced from reputable outlets') shift VLM accuracy by large margins, with adversarial hints dropping Qwen2.5-VL-32B's non-misleading accuracy from 95.9 to 3.4.
• Stronger visual fidelity (GPT-image-1 vs FLUX) raises detection accuracy for several VLMs, and GPT-image-1 safety guardrails could be bypassed to generate manipulations for all 200 test cases.

Limitations

• Open-ended intent articulation and evaluation is not addressed; LLM-as-judge is proposed only as future work.
• Societal impact of misinformation is not measured, lacking audience engagement signals.
• Human annotators disagreed most on subtle misleading samples, indicating intent labels are partly subjective.

Experimental setup

Datasets: DeceptionDecoded (constructed from VisualNews) VisualNews MMFakeBench Fakeddit FakeNewsNet

Benchmarks: DeceptionDecoded MMFakeBench Fakeddit FakeNewsNet

Metrics: accuracy F1 macro-F1 Fleiss' kappa

Baselines: o4-mini Claude-3.7-Sonnet Gemini-2.5-Pro GPT-4o Claude-3.5-Sonnet Gemini-1.5-Pro GPT-4o-mini Claude-3.5-Haiku Qwen2.5-VL-72B Qwen2.5-VL-32B Llama-3.2-Vision-11B InternVL-8B

Builds on: VisualNews NewsCLIPpings DGM4 MMFakeBench

Closest papers by meaning

• The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contexts (CVPR 2026 · similarity 0.86)
• Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models (ICML 2026 · similarity 0.86)
• SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection (CVPR 2024 · similarity 0.86)
• Reinforced Adaptive Knowledge Learning for Multimodal Fake News Detection (AAAI 2024 · similarity 0.85)
• Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation (CVPR 2025 · similarity 0.85)
• [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning (NeurIPS 2026 · similarity 0.85)