Research map / Vision-language and multimodal / Multimodal LLMs
Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Hao Fang, Jinyu Li, Jiawei Kong, Tianqu Zhuang, Kuofeng Gao, Bin Chen, Shu-Tao Xia
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Investigates why multimodal large reasoning models hallucinate more and proposes a reasoning-centric training framework to mitigate hallucinations.
Problem
Multimodal large reasoning models (MLRMs) remain prone to hallucinations, and effective mitigation methods are underexplored.
Approach
Attributes MLRM hallucinations to reasoning mechanisms that increase reliance on language priors, and proposes a reasoning-centric training framework to counter this.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add C3PO (CoT compression + contrastive preference optimization) | Orsta-R1-7B · CHAIR | CHAIR_S: 40.7 -> 35.3 | Improved |
| apply RLAIF-V baseline to MLRM | R1-Onevision-7B · CHAIR | CHAIR_S: 36.7 -> 39.0 | Hurt |
| apply OPA-DPO baseline to MLRM | Orsta-R1-7B · CHAIR | CHAIR_S: 40.7 -> 41.7 | Hurt |
| add C3PO | MM-Eureka-7B · POPE Random | accuracy (%): 84.12 -> 86.22 | Improved |
| add C3PO | MM-Eureka-7B · AMBER discriminative | accuracy (%): 83.27 -> 85.01 | Improved |
| add reasoning-enhanced preference loss L_RE (ablation) | C3PO · CHAIR | CHAIR_S: 37.3 -> 35.3 | Improved |
| add standard DPO loss L_DPO over joint output (ablation) | C3PO · CHAIR | CHAIR_S: 39.7 -> 35.3 | Improved |
| add anchor regularization L_Anc (ablation) | C3PO · CHAIR | CHAIR_S: 37.7 -> 35.3 | Improved |
| add multimodal hallucination-inducing negatives (ablation) | C3PO · GPT-4 assisted eval | SHR: 28.4 -> 26.3 | Improved |
| add preference optimization after SFT on compressed CoT (ablation) | Orsta-R1-7B · CHAIR / GPT-4 eval | CHAIR_S and SHR: reductions of 4.4 and 2.9 in C_S and SHR | Improved |
| SFT on compressed CoT alone (no DPO) (ablation) | R1-Onevision-7B · GPT-4 assisted eval | SHR: w/o DPO 36.7 vs C3PO 37.9 | Mixed |
| prune CoT tokens (gamma < 1.0) instead of no pruning (ablation) | Orsta-R1-7B · CHAIR and GPT-4 eval | CHAIR / SHR | Improved |
Surprising findings
- Introducing reasoning mechanisms exacerbates models' reliance on language priors, which the abstract identifies as a cause of hallucinations in MLRMs.
Closest papers by meaning
- More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models (NeurIPS 2025 · similarity 0.94)
- Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models (CVPR 2026 · similarity 0.94)
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (NeurIPS 2025 · similarity 0.93)
- Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning (AAAI 2025 · similarity 0.93)
- Understanding and Mitigating Hallucinations in Multimodal Chain-of-Thought Models (CVPR 2026 · similarity 0.93)
- Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models (NeurIPS 2025 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
