Research map / Vision-language and multimodal / Multimodal LLMs
Does Reasoning Improve Seeing? Understanding When Vision-Language Models Benefit from Thinking
Jing Bi, Luchuan Song, Dingxin Zhang, Pinxin Liu, Guangyu Sun, Lianggong Bruce Wen, Weidong Cai, Chen Chen, Chenliang Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Investigates whether VLMs encode meta-cognitive signals indicating when explicit reasoning (Thinking mode) helps versus direct Instruct answering, to enable adaptive test-time inference.
Problem
Practitioners lack principled ways to decide when reasoning improves vision-language model performance and how much test-time computation to spend.
Approach
Derives oracle labels for reasoning helpfulness (whether Thinking mode beats Instruct mode on a sample) and probes whether VLM internal representations encode these signals for adaptive inference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Thinking-model representations instead of Instruct for linear probing of reasoning helpfulness (ablation) | linear probe on final-layer last-token embedding · MMMU-val900, InternVL-8B | MSE (r_help): 0.168 -> 0.119 | Improved |
| use Thinking-model representations instead of Instruct for probing desired length (ablation) | linear probe on last-token embedding · MMMU-val900, Qwen3-VL-32B | MAE (l_opt): 39.1 -> 31.8 | Improved |
| increase model size from 8B to 32B (ablation) | Thinking-probe for r_help · MMMU-val900, Qwen3-VL | Spearman rho: 0.447 -> 0.531 | Improved |
| scale length-control heads (alpha 0.5 to 2.0) (ablation) | Thinking VLMs · MMMU, InternVL-8B-Thinking | mean generation length (tokens): 139 -> 521 | Improved |
| scale length-control heads (alpha 0.5 to 2.0) (ablation) | Thinking VLMs · MMMU, InternVL-8B-Thinking | accuracy: -0.8% | No clear effect |
| amplify low-layer reasoning heads while suppressing perception heads (ablation) | InternVL-8B-Thinking · MMMU perception-dominant cohort | accuracy change: +4.1% | Improved |
| amplify high-layer reasoning heads (ablation) | InternVL-8B-Thinking · MMMU reasoning-dominant cohort | accuracy change: +5.3% | Improved |
| depth-controlled reasoning head steering (ablation) | InternVL-8B-Thinking · MMMU overall | overall accuracy change: +1.2% | Improved |
| head-scaling steering applied to other benchmarks (ablation) | InternVL-8B-Thinking · MathVista, ScienceQA, ChartQA, A-OKVQA | accuracy change: +2.9% to +4.9% | Improved |
| increase number of stochastic passes K from 5 to 10 for labels (ablation) | oracle label estimation · Qwen3-VL-4B | corr(r_K, r_40): 0.885 -> 0.962 | Improved |
| vary head-set size K from 8 to 20 (ablation) | head attribution · MMMU, averaged across models | Jaccard overlap: 0.71 -> 0.68 for J(Sr,Sd) | No clear effect |
Closest papers by meaning
- From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.93)
- SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.91)
- Caption This, Reason That: VLMs Caught in the Middle (NeurIPS 2025 · similarity 0.91)
- Bring Reason to Vision: Understanding Perception and Reasoning through Model Merging (ICML 2025 · similarity 0.90)
- When to Think and When to Look: Uncertainty-Guided Lookback (CVPR 2026 · similarity 0.90)
- More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
