Research map / Vision-language and multimodal / Multimodal LLMs

SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds SpatialLadder-26k, a multimodal spatial reasoning dataset from ScanNet and SR-91k, and trains Qwen2.5-VL-3B with a three-stage progressive pipeline (localization SFT, multi-dimensional spatial SFT, GRPO with verifiable rewards) to reach state-of-the-art spatial reasoning results.

Problem

VLMs struggle with spatial reasoning; existing methods learn spatial reasoning directly without a hierarchical foundation of perception and understanding, and existing datasets are fragmented across 2D images, 3D scenes and video.

Approach

Progressive curriculum that first trains object localization for perceptual grounding, then multi-dimensional spatial QA across single-image, multi-view and video data, and finally applies GRPO with format and task-specific accuracy rewards, all on a unified 26k-sample dataset built from ScanNet 3D metadata.

Key results

• SpatialLadder-3B reaches 62.3% overall on in-domain benchmarks, a 23.4-point gain over Qwen2.5-VL-3B and above GPT-4o (41.5) and Gemini-2.0-Flash (52.2).
• On VSI-Bench SpatialLadder-3B scores 45.7%, comparable to Spatial-MLLM-4B's 47.3%, using a standard VLM architecture.
• Out-of-domain overall accuracy is 50.8% vs 43.6% for the base model and 48.1% for GPT-4o, with a 16.5-point gain on ViewSpatial-Bench person-perspective.

Surprising findings

• Removing Stage 2 causes a 9.4% drop, larger than removing Stage 1 (1.8%) or Stage 3 (2.1%).
• Semantic entropy first rises from 1.24 to 1.47 during SFT stages and then falls to 0.66 during RL.

Limitations

• Relies on template-generated QA from ScanNet metadata, which may limit linguistic and scenario diversity.
• Stage-wise ablation and some reported numbers are presented mainly in figures; several result tables are only partially detailed in the text.
• Evaluation on SPBench-SI/MV uses benchmarks built by the same pipeline, so in-domain gains may partly reflect distribution match.

Experimental setup

Datasets: ScanNet SR-91k VSI-Bench SpatialLadder-26k

Benchmarks: VSI-Bench SPBench-SI SPBench-MV CV-Bench SPAR-Bench ViewSpatial-Bench

Metrics: accuracy average accuracy over MCQ and numerical questions visual attention IoU semantic entropy

Baselines: GPT-4o Gemini-2.0-Flash InternVL-2.5-4B InternVL-2.5-8B Kimi-VL-A3B LLaVA-OneVision-7B Qwen2.5-VL-7B SpaceR-7B VILASR-7B Video-R1 Spatial-MLLM-4B

Builds on: GRPO Qwen2.5-VL VLM-R1 VSI-Bench

Closest papers by meaning

• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.94)
• SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.93)
• Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.93)
• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.92)
• From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
• InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)