Research map / Vision-language and multimodal / Multimodal LLMs
Mitigating Low-Quality Reasoning in MLLMs: Self-Driven Refined Multimodal CoT with Selective Thinking and Step-wise Visual Enhancement
Chongjun Tu, Peng Ye, Dongzhan Zhou, Tao Chen, Wanli Ouyang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-driven refined multimodal chain-of-thought approach for MLLMs that selectively enables thinking on hard queries and strengthens visual grounding at each reasoning step to reduce overthinking and low-quality reasoning.
Problem
Multimodal chain-of-thought (MCoT) in MLLMs suffers from low-quality reasoning: overthinking on simple queries and inefficient use of visual information, wasting computation.
Approach
MLLMs inherently can distinguish simple from difficult queries; the method exploits this to selectively trigger thinking and applies step-wise visual enhancement during reasoning, self-driven without external supervision (details not stated in the abstract).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add selective thinking (entropy-based direct answer vs reasoning) (ablation) | SDR-MCoT · CoMT/MMStar/HallusionBench | avg token consumption: 212.91 -> 74.07 | Improved |
| add selective thinking (ablation) | SDR-MCoT · HallusionBench | accuracy: 63.76 -> 67.39 | Improved |
| add step-wise visual enhancement (ablation) | SDR-MCoT · HallusionBench | accuracy: 63.76 -> 67.76 | Improved |
| add step-wise visual enhancement (ablation) | SDR-MCoT · token consumption | tokens per answer: 212.91 -> 216.57 | No clear effect |
| combine selective thinking and visual enhancement (ablation) | SDR-MCoT · MMStar | accuracy: 53.62 -> 56.39 | Improved |
| apply zero-shot CoT to base MLLM | Qwen2-VL-7B · average over 8 benchmarks | average accuracy: 59.00 -> 57.34 | Hurt |
| apply zero-shot CoT to base MLLM | InternVL3-8B · average over 8 benchmarks | average accuracy: 70.28 -> 65.90 | Hurt |
| apply SDR-MCoT instead of CoT | Qwen2-VL-7B · average over 8 benchmarks | average accuracy: 57.34 -> 63.69 | Improved |
| apply SDR-MCoT instead of CoT | InternVL3-8B · average over 8 benchmarks | average accuracy: 65.90 -> 72.87 | Improved |
| apply SDR-MCoT | LLaVA-1.5-13B · average over 8 benchmarks | average accuracy: 46.40 -> 48.57 | Improved |
| apply visual-supplementation CoT (CCoT, DDCoT) | Qwen2-VL-7B · average over 8 benchmarks | average accuracy and tokens: CCoT 57.30 avg, 920.35 tokens | Hurt |
Surprising findings
- MLLMs possess inherent ability to distinguish simple from difficult queries, per the abstract's discovery claim
Closest papers by meaning
- Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training (ICML 2026 · similarity 0.93)
- AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.92)
- MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.92)
- TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding (ICML 2026 · similarity 0.92)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.92)
- Look-Back: Implicit Visual Re-focusing in MLLM Reasoning (AAAI 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
