Research map / Vision-language and multimodal / Multimodal LLMs

When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MIRA, a 546-example benchmark for visual chain-of-thought where intermediate sketches/diagrams are needed, and evaluates MLLMs under direct, text-CoT, and simulated visual-CoT (human-annotated intermediate images) settings.

Problem

Existing multimodal reasoning benchmarks rarely require generating or using intermediate visual images; models reason almost entirely in text, which is lossy for spatial, geometric, and physical problems.

Approach

A benchmark of 20 task types across geometry, physics, abstract puzzles, and causal transformations, each paired with human-annotated step-wise visual clues, and a three-level protocol (direct, text-CoT, simulated visual-CoT) that measures the contribution of intermediate visuals.

Key results

• GPT-5 reaches only 16.5% overall accuracy under direct input, and no evaluated model exceeds 20% in that setting.
• Simulated Visual-CoT with human-annotated intermediate images raises GPT-5-mini from 13.7% to 23.2% overall, and Physics accuracy for Gemini 2.5 Pro rises from 41.1% to 59.5%.
• Text-CoT reduces accuracy for Gemini 2.5 Pro and o3 by relative 18.3% and 14.0%.

Surprising findings

• Text-CoT degrades performance for strong models such as Gemini 2.5 Pro and o3 on MIRA.
• Pass@k and majority voting yield smaller gains for stronger models, suggesting failures reflect missing capability rather than sampling noise.

Limitations

• Visual-CoT is simulated with human-annotated intermediate images rather than models generating their own visuals.
• Benchmark is small (546 examples across 20 tasks), so per-task results are noisy.
• Open-weight models evaluated are limited to a few large flagship models and two small unified models.

Experimental setup

Datasets: MIRA

Benchmarks: MIRA

Metrics: accuracy pass@k majority voting

Baselines: GPT-5 Gemini 2.5 Pro o3 Claude 4 Opus Qwen2.5-VL (72B) GLM 4.5V Bagel Janus-Pro

Builds on: chain-of-thought prompting Visual CoT Visual Sketchpad majority voting

Closest papers by meaning

• Imagine While Reasoning in Space: Multimodal Visualization-of-Thought (ICML 2025 · similarity 0.92)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.92)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.92)
• Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.91)
• MentisOculi: Revealing the Limits of Reasoning with Mental Imagery (ICML 2026 · similarity 0.91)
• Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.91)