Research map / Vision-language and multimodal / Multimodal LLMs

Drifting Away from Truth: GenAI-Driven News Diversity Challenges LVLM-Based Misinformation Detection

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DriftBench, a 16,000-instance benchmark that diversifies NewsCLIPpings image-text news with GenAI to test how LVLM-based misinformation detectors degrade under multi-level drift and adversarial evidence contamination.

Problem

LVLM-based multimodal misinformation detection is fragile when GenAI produces highly varied news content, causing model-level misperception drift and evidence-level drift in retrieved evidence, plus vulnerability to adversarially injected evidence.

Approach

Taxonomy of GenAI-driven news diversity (controlled vs open-ended, real vs fake, image/text variants) implemented as an automated GPT-4o and FLUX.1 pipeline, with three evaluation tasks: performance under diversity, robustness to malicious evidence injection, and LVLM-as-judge reasoning analysis.

Key results

• Diversification drops average F1 by 14.8% across evaluated LVLM-based detectors on DriftBench.
• Malicious evidence contamination lowers GPT-4o-mini accuracy on DI_OT from 64.4 to 44.6 and Qwen-VL from 63.7 to 37.2.
• Human validation reports 98.6% semantic preservation accuracy (Fleiss' kappa 0.872) for diversified real instances.

Surprising findings

• Text diversification has minimal impact while image diversification drives most of the performance decline.
• Task-specific detectors CMIE and LEMMA are more stable under contamination than vanilla LVLMs, yet MMD-specific models show larger drops on fake instances.

Limitations

• Evaluation covers only six detectors and three vanilla LVLMs, with Qwen-VL-Plus as the sole open-source vanilla model.
• Human quality check uses only 40 samples per diversification type and only real-instance variants are fully verifiable.
• Malicious evidence is generated by GPT-4o, so attack strength is limited to one generator.

Experimental setup

Datasets: NewsCLIPpings

Benchmarks: DriftBench

Metrics: accuracy precision recall F1 explanation quality scores

Baselines: GPT-4o-mini Claude-3.7-Sonnet Qwen-VL-Plus CMIE SNIFFER LEMMA

Builds on: NewsCLIPpings LEMMA SNIFFER CMIE

Closest papers by meaning

• MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs (ICLR 2025 · similarity 0.87)
• Bad Actor, Good Advisor: Exploring the Role of Large Language Models in Fake News Detection (AAAI 2024 · similarity 0.86)
• SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection (CVPR 2024 · similarity 0.85)
• Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection (NeurIPS 2025 · similarity 0.85)
• Fact-R1: Towards Explainable Video Misinformation Detection with Deep Reasoning (NeurIPS 2025 · similarity 0.85)
• LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models (ICLR 2025 · similarity 0.85)