Research map / Vision-language and multimodal / Multimodal LLMs

More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that multimodal reasoning models hallucinate more on perception tasks than their non-reasoning Qwen2.5-VL backbones, attributes this to reduced visual attention that worsens with longer reasoning, and introduces the RH-AUC metric and RH-Bench benchmark to measure the reasoning-hallucination balance.

Problem

Multimodal reasoning models (trained with SFT and/or RL for long chain-of-thought) gain reasoning accuracy but may lose visual grounding, increasing hallucination on perception-focused tasks; existing metrics at a fixed generation length do not capture this trade-off.

Approach

Combines attention analysis, controlled reasoning-length manipulation (budget forcing, test-time "Wait" extension, latent state steering) and a new AUC-style metric (RH-AUC) over the curve of reasoning accuracy versus perception accuracy across reasoning lengths, plus a 1000-sample diagnostic benchmark.

Key results

• Ocean-R1-7B achieves the highest RH-AUC of 0.63 on RH-Bench, while LLM-R1-3B and R1-OneVision-7B score 0.46.
• Across eight reasoning models versus Qwen2.5-VL backbones at 3B and 7B, all reasoning models show higher hallucination rates on five perception benchmarks.
• RL-only models generally achieve better RH-AUC than SFT+RL models, and the paper reports that more perception data does not reliably improve balance.

Measured effects reported in the paper

Surprising findings

• Reasoning models hallucinate more than their non-reasoning Qwen2.5-VL backbones on perception benchmarks at both 3B and 7B scales.
• More visual perception training data does not necessarily improve the reasoning-perception balance; R1-OneVision with 80k perception samples scores low.

Limitations

• Evaluation limited to models built on the Qwen2.5-VL backbone, which may limit generalization.
• Training-data conclusions are observational, based on technical reports rather than controlled retraining.
• Open-ended scoring relies on GPT-4o judgments with a threshold for hallucination.

Experimental setup

Datasets: MathVision MathVista MMMU ScienceQA MMHalu MMVP HallusionBench VMCBench MMEval-Pro Bingo MMHAL-Bench

Benchmarks: RH-Bench MMVP MMEval-Pro VMCBench Bingo MMHAL MathVista

Metrics: accuracy hallucination rate RH-AUC average response length

Baselines: Qwen2.5-VL-3B Qwen2.5-VL-7B LLM-R1-3B Curr-ReFT-3B Ocean-R1-3B R1-OneVision-7B ThinkLite-VL-7B OpenVLThinker-7B MM-Eureka-7B MM-R1-7B Ocean-R1-7B

Builds on: GRPO Qwen2.5-VL DeepSeek-R1

Closest papers by meaning

• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (NeurIPS 2025 · similarity 0.94)
• Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models (CVPR 2026 · similarity 0.93)
• Reallocating Attention Across Layers to Reduce Multimodal Hallucination (CVPR 2026 · similarity 0.93)
• Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models (NeurIPS 2025 · similarity 0.93)
• Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning (AAAI 2025 · similarity 0.93)
• Understanding and Mitigating Hallucinations in Multimodal Chain-of-Thought Models (CVPR 2026 · similarity 0.93)