Research map / Vision-language and multimodal / Multimodal LLMs

Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CSMR, a training-free framework where an LLM reasoning core iteratively queries an independent VLM perception module for textualized visual evidence and decides when to stop, outperforming text-conversion and unified VLM CoT baselines on M3CoT, ScienceQA, and LLaVA-W.

Problem

Multimodal reasoning methods either convert images to static text before reasoning (losing fine-grained evidence) or reason in a unified VLM space where linguistic priors weaken visual grounding.

Approach

A text-only LLM reasoning core maintains an explicit reasoning state and dynamically issues visual queries to a separate perception VLM, deciding both when to query and when to terminate, instead of one-shot captioning or fused multimodal reasoning.

Key results

• CSMR reaches 45.7 accuracy on M3CoT versus 44.1 for ICoT and 39.0 for DDCoT with Qwen2-VL-7B perception and Qwen2-7B reasoning.
• On ScienceQA, CSMR scores 78.2 versus 71.9 for DDCoT and 56.8 for ICoT under the same backbone setup.
• With Qwen3-VL-8B perception and Qwen3-8B reasoning, CSMR achieves 53.6 on M3CoT versus 50.3 for DDCoT; a fixed seven-step variant drops to 42.7 accuracy.

Surprising findings

• Qwen3-VL-8B answers ScienceQA subset questions at 57.33% accuracy with no image and no hint, far above chance, which the authors take as evidence of linguistic-prior reliance.
• Fixing the number of CRC-PVP interaction steps to seven reduces accuracy relative to flexible termination (42.7 vs 45.7 on M3CoT).

Limitations

• Multi-model collaboration and dynamic querying add substantial inference overhead (about 24 s per sample vs 12 s for single-query and 5 s for caption on M3CoT).
• Evaluated on only three benchmarks with a single backbone family in the main table; the reported gains narrow when the perception model is stronger (3.3 vs 4.8 points over DDCoT).
• The hallucination analysis relies on GPT-5 as an automatic judge over 200 sampled instances.

Experimental setup

Datasets: M3CoT ScienceQA LLaVA-Bench In-the-Wild

Benchmarks: M3CoT ScienceQA LLaVA-Bench In-the-Wild

Metrics: accuracy ROUGE-L seconds per sample hallucination rate

Baselines: No-CoT Caption Multimodal CoT CCoT SCAFFOLD ICoT DDCoT

Builds on: DDCoT Multimodal CoT ICoT

Closest papers by meaning

• Look-Back: Implicit Visual Re-focusing in MLLM Reasoning (AAAI 2026 · similarity 0.92)
• See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs (CVPR 2026 · similarity 0.92)
• VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning (ICML 2026 · similarity 0.91)
• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.91)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.91)
• Latent Implicit Visual Reasoning (CVPR 2026 · similarity 0.91)