Research map / Vision-language and multimodal / Multimodal LLMs

Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hallucination-as-Cue, a framework that applies modality-specific corruptions (blank image, random image, text removal) during RL post-training and evaluation of Qwen2.5-VL models with GRPO, showing corrupted RL training still improves multimodal math reasoning benchmarks.

Problem

Unclear whether RL post-training (GRPO) of multimodal LLMs actually learns from visual information or mainly strengthens text-based reasoning within the LLM.

Approach

Deliberately induce hallucination by corrupting or removing visual/textual inputs during both RL training and inference, and use the resulting accuracy and behavior to diagnose how much RL relies on visual grounding versus textual priors, treating hallucinated correct answers as rewarded "hallucination-inductive rewards".

Key results

• Qwen2.5-VL-3B GRPO-BI (blank image RL training) reaches 42.25% average vs 39.72% base and 44.74% standard GRPO across four benchmarks.
• Qwen2.5-VL-7B GRPO-RI (random image training) reaches 54.23% average, above standard GRPO at 53.52%.
• BI corruption at inference drops Qwen2.5-VL-7B average from 50.89% to 29.20%, showing models still rely heavily on visual input.

Surprising findings

• RL training with blank images (no visual information) still improves reasoning accuracy over the base model.
• For Qwen2.5-VL-7B, random-image RL training slightly surpasses standard GRPO on average.
• BI-corrupted inference slightly increases MathVision accuracy for Qwen2.5-VL-3B (18.19% to 18.91%).

Limitations

• Mechanisms behind the observed behaviors are not fully explained, as acknowledged by the authors.
• Study limited to mainstream GRPO-based training; other reasoning paradigms such as latent-space reasoning are not examined.
• Evaluation focuses on visual mathematical reasoning benchmarks with only Qwen2.5-VL models; differences between corruption types are often small with single runs reported.

Experimental setup

Datasets: Geometry3K MMR1-V0 CLEVR MathVision MathVerse MathVista WeMath

Benchmarks: MathVision MathVerse MathVista WeMath Geometry3K

Metrics: accuracy

Baselines: Qwen2.5-VL-3B Qwen2.5-VL-7B GRPO

Builds on: GRPO DeepSeek-R1 EasyR1

Closest papers by meaning

• More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models (NeurIPS 2025 · similarity 0.93)
• Ground What You See: Hallucination-Resistant MLLMs via Caption Feedback, Diversity-Aware Sampling, and Conflict Regularization (AAAI 2026 · similarity 0.93)
• See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs (CVPR 2026 · similarity 0.92)
• Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models (NeurIPS 2025 · similarity 0.92)
• Learning to Reason for Hallucination Span Detection (ICLR 2026 · similarity 0.91)
• Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning (AAAI 2025 · similarity 0.91)