Research map / Vision-language and multimodal / Multimodal LLMs

Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematic study of test-time compute for VLMs showing feature heuristics fail and voting gives modest gains; proposes entropy-based selection (ETTC) that beats majority voting and the best single model in multi-model ensembles.

Problem

Test-time compute strategies from LLMs (best-of-N with lexical features, majority voting) have unclear benefits for vision-language models, where perception errors dominate and predictions are highly correlated.

Approach

Theoretically links voting gains to prediction dependency (NMI and correctness correlation), and proposes Entropy-based TTC, which selects the top-1 answer from the source with lowest normalized predictive entropy; this reduces to voting in single-model settings and lets confident smaller models override larger ones in ensembles.

Key results

• On the cross-family ensemble (LLaMA, Pixtral, Gemma, Qwen-7B), ETTC averages 66.56% vs 63.75% for majority voting and 57.29% for the average single model.
• On the Qwen same-family ensemble (3B/7B/32B/72B), ETTC averages 71.68% vs 68.84% for voting and surpasses the strongest model Qwen-72B (69.90%).
• On MathVista with Qwen 3B+72B, voting gets 73.15% while ETTC reaches 84.81%, exceeding the 72B single model's 80.58%.

Surprising findings

• Feature-based CoT selection (pivot words, length, composite features) fails to beat vanilla CoT, and voting gives no gain without CoT prompting.
• Smaller models (e.g., 3B) can improve larger 72B models when combined via entropy-based selection, while voting is dragged down.

Limitations

• Entropy-accuracy monotonicity is an assumption that holds only in aggregate, and models can be confidently wrong or miscalibrated.
• Voting trends are inconsistent on MathVision, where low accuracy (~30%) makes dependency measures unreliable.
• Evaluation is restricted to multiple-choice benchmarks; supervised ETTC requires labeled data.

Experimental setup

Datasets: MathVista MathVision TQA ScienceQA MMStar MMMU

Benchmarks: MathVista MathVision TQA ScienceQA MMStar MMMU ARC-Easy MMLU-Pro

Metrics: accuracy

Baselines: CoT Pivot Word CoT Length Feature-All Majority Voting Average single model

Builds on: self-consistency chain-of-thought prompting best-of-N selection

Closest papers by meaning

• The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.91)
• SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.90)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.90)
• When to Think and When to Look: Uncertainty-Guided Lookback (CVPR 2026 · similarity 0.90)
• Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.90)
• UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling (NeurIPS 2024 · similarity 0.90)