Research map / Vision-language and multimodal / Multimodal LLMs

The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes S-BOED, a Bayesian experimental design view of active visual reasoning under a perceptual bandwidth bottleneck, and FOVEA, a training-free crop-refinement procedure for VLM tool calls that scores candidate crops by VLM yes/no resolvability probes.

Problem

High-resolution visual reasoning in VLMs is limited by a fixed visual-token budget, so small objects, text and fine details are lost in global views; existing crop/tool policies are heuristic and lack a decision-theoretic objective for which view to acquire.

Approach

Formalises active visual evidence acquisition as sequential BOED and derives a coverage-resolution proxy objective (posterior mass in crop times resolution probability); instantiates it training-free by refining the VLM's crop proposals with candidate perturbations scored by self-reported resolvability (greedy, MCMC, or one-step lookahead).

Key results

• On the 30B backbone FOVEA reaches 77.7% mean across four benchmarks versus 75.1% for ReAct and 73.3% for Direct on Qwen3-VL-30B-A3B-Instruct.
• On the 8B backbone FOVEA raises mean accuracy from 72.5% (ReAct) and 70.9% (Direct) to 74.9%.
• On the 50/150-question Remote Sensing subset FOVEA-Lookahead reaches 54.7% versus 45.1% for ReAct, at roughly 9.5x input tokens.

Surprising findings

• Lookahead gains come at high cost: FOVEA-Lookahead uses about 9.5x input tokens and 55x output tokens relative to ReAct.
• RAP scores highest on V* (86.4%) and HR-Bench 8K (80.6%) yet has the lowest mean among the 30B-backbone methods on MME-RealWorld-lite (40.8%) and CV-Bench (72.2%).

Limitations

• Relies on an ideal-observer approximation; backbone recognition errors persist even with oracle crops.
• Proposal-limited: if the relevant region never enters the candidate pool, local refinement cannot recover it.
• Resolvability probing and search add substantial inference-time overhead.

Experimental setup

Datasets: HR-Bench MME-RealWorld-Lite V*Bench CV-Bench

Benchmarks: MME-RealWorld-Lite CV-Bench V*Bench HR-Bench (4K) HR-Bench (8K)

Metrics: accuracy mean accuracy tokens per query

Baselines: Direct ReAct Agent RAP Thyme GPT-5 Gemini 2.5 Flash

Builds on: ReAct Bayesian optimal experimental design Thyme

Closest papers by meaning

• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.90)
• SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.90)
• Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning (ICML 2026 · similarity 0.90)
• VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs (CVPR 2026 · similarity 0.90)
• Act2See: Emergent Active Visual Perception for Video Reasoning (CVPR 2026 · similarity 0.90)
• VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs (NeurIPS 2025 · similarity 0.90)