Research map / Vision-language and multimodal / Multimodal LLMs

SpatialViz-Bench: A Cognitively-Grounded Benchmark for Diagnosing Spatial Visualization in MLLMs

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SpatialViz-Bench, a programmatically generated multimodal benchmark of 1,180 problems across 12 tasks in 4 spatial visualization sub-abilities, and evaluates 27 MLLMs, finding low accuracy and a CoT paradox in open-source models.

Problem

Existing multimodal benchmarks rarely assess spatial visualization (mentally manipulating unseen structures), and the few that do rely on web-sourced IQ-test or competition problems that risk data contamination and are poorly categorized.

Approach

Decompose spatial visualization into four cognitively grounded sub-abilities (mental rotation, folding, visual penetration, animation) and generate most test items programmatically with Python and FreeCAD, with controlled difficulty and explained distractors, to reduce contamination and allow continuous expansion.

Key results

• Gemini-2.5-pro achieved the top overall accuracy of 44.66% versus 82.46% for human participants and ~25% random baseline.
• Chain-of-thought lowered accuracy for several open-source models (e.g., Kimi-VL-A3B-Instruct, Qwen2.5-Omni-7B, SAIL-VL-1.6-8B) with statistically significant negative impact, while Claude-3.5-sonnet improved significantly.
• Text-only Qwen2.5-72B-Instruct scored 25.86%, near random, showing the benchmark is vision-dependent.

Surprising findings

• Chain-of-thought prompting degrades accuracy for several open-source MLLMs, a 'CoT paradox'.
• Scaling Qwen2.5-VL from 7B to 72B barely changes the dominant perceptual and spatial transformation error patterns.

Limitations

• Mechanical System task (80 items) was manually designed, so it is not programmatically scalable.
• Most evaluated models are queried zero-shot with multiple-choice formats; human baseline uses only 8 graduate students on a 72-problem subset.

Experimental setup

Datasets: DeepCAD

Benchmarks: SpatialViz-Bench

Metrics: accuracy Wilson 95% confidence intervals p-value (CoT ablation)

Baselines: Human Random Qwen2.5-72B-Instruct (text-only) GPT-4o o1 Claude-3.7-sonnet Gemini-2.5-flash

Builds on: CLEVR MM-IQ MathVision DeepCAD

Closest papers by meaning

• MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence (ICLR 2026 · similarity 0.91)
• Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests (NeurIPS 2026 · similarity 0.91)
• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.90)
• Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.90)
• SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.90)
• Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)