Research map / Vision-language and multimodal / Multimodal LLMs

Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Spatial-DISE, a cognitively grounded 2x2 (intrinsic/extrinsic × static/dynamic) benchmark of 559 VQA pairs plus a 12K-pair Blender-generated training set, evaluates 32 VLMs against humans, and shows large gaps especially on multi-step dynamic tasks.

Problem

Existing spatial reasoning benchmarks for VLMs lack a systematic cognitive framework, focus on static scenes, and offer too little dynamic (mental transformation) data at scale.

Approach

A DISE taxonomy (Intrinsic/Extrinsic × Static/Dynamic) organizes 10 psychometric-inspired spatial tasks, and a seeded Blender pipeline with tiered distractor generation produces verifiable synthetic VQA for training and evaluation.

Key results

• Best model Doubao1.5-VL-thinking reaches 42.0% on Spatial-DISE Bench versus 76.8% human accuracy, with random guessing at 24.8%.
• Fine-tuning Qwen2.5-VL-7B with LoRA on Spatial-DISE-12K raises Spatial-DISE Bench accuracy from 26.1% to 47.0%, and improves external benchmarks such as OmniSpatial (21.8% to 34.0%).
• Layout ablation on Qwen2.5-VL-7B shows near-identical accuracy for merged (26.1%) vs separate-image (24.9%) inputs, supporting reasoning rather than perception as the bottleneck.

Surprising findings

• Doubao1.5-VL-thinking surpasses human accuracy on Extrinsic-Dynamic tasks (61.9% vs 61.1%).
• Several models score higher on dynamic than static tasks, suggesting static comprehension is not a prerequisite for dynamic reasoning.

Limitations

• Spatial-DISE-12K is entirely synthetic, so a sim-to-real gap may persist.
• Only a stratified sample of 1,000 items received human quality control; the rest were checked programmatically.
• Error analysis uses an LLM-proposed label verified by a human annotator on only 200 incorrect responses, which may bias labels.

Experimental setup

Datasets: Spatial-DISE-12K real-world psychometric and aptitude test items (1180 VQA pairs collected) CVBench SAT SPACE OmniSpatial VSI-Bench MCQ

Benchmarks: Spatial-DISE Bench CVBench SAT SPACE OmniSpatial VSIBench_MCQ

Metrics: accuracy 95% confidence interval per-DISE-quadrant accuracy

Baselines: Random Guessing Human Performance SpaceThinker SpaceOM SpaceR VLM-R1 LMM-R1 LLaVA-CoT

Builds on: VLMEvalKit LoRA

Closest papers by meaning

• SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.95)
• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.94)
• InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.93)
• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.93)
• SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.93)
• SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.93)