Research map / Vision-language and multimodal / Multimodal LLMs

Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Spatial-IQ, a hierarchical diagnostic benchmark decomposing 3D object counting in stacked blocks into 9 sub-tasks, and shows that SFT with CoT over sub-tasks plus RLVR (DAPO) improves Qwen2.5-VL counting accuracy.

Problem

MLLM benchmarks evaluate spatial reasoning as black boxes with end-task scores, so it is unclear whether failures are perceptual or cognitive and whether models succeed through the intermediate structure humans use.

Approach

Decompose a target task (object counting in stacked 3D blocks) into a dependency hierarchy of sub-tasks grounded in developmental psychology, evaluate models across text, MCQ and image-editing formats, and use hierarchical chain-of-thought SFT plus RLVR with the decomposition as a training signal.

Key results

• Humans reach 82.1% on Object Counting while the best text model (Qwen3.5-27B) reaches 17.7%.
• DAPO-tight 32B reaches 62.6% Object Counting vs 2.9% zero-shot Qwen2.5-VL-32B and 40.1% for SFT-plain 32B.
• SFT-plain collapses to 0% on Internal Referential Chain, Hidden Support Hierarchy and Summation Mechanism while SFT-CoT preserves them.

Surprising findings

• Under 5-choice MCQ, every evaluated model performs at chance on Object Counting despite non-trivial sub-task accuracy.
• SFT-plain reaches comparable Object Counting accuracy to SFT-CoT at 7B while bypassing the sub-task decomposition entirely.

Limitations

• Cross-modality comparisons are not apples-to-apples because response formats change answer-selection demands.
• Training and evaluation restricted to textured cubes in the text modality.
• Sub-tasks operationalized slightly differently across formats.

Experimental setup

Datasets: Spatial-IQ

Benchmarks: Spatial-IQ

Metrics: accuracy conditional accuracy on hierarchy relations chance-adjusted accuracy

Baselines: Gemini 3 Pro GPT 5.4 Claude Opus 4.6 Qwen3.5-27B Kimi K2.5 GLM 4.6 VLA-0 Qwen3.5-3B Gemini 3.1 Flash Image Qwen-Image-Edit HunyuanImage-Instruct SFT-plain

Builds on: DeepSeek-R1 GRPO DAPO Piaget developmental stages

Closest papers by meaning

• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.92)
• SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.91)
• Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.91)
• From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs (CVPR 2026 · similarity 0.91)
• Scaling Spatial Intelligence with Multimodal Foundation Models (CVPR 2026 · similarity 0.91)
• 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.91)