Research map / Vision-language and multimodal / Multimodal LLMs
SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SpatialReasoner, a Qwen2.5-VL-7B-based LVLM that reasons over explicit 3D representations (3D locations and orientations in calibrated camera space) via SFT on synthetic SR-CoT data followed by GRPO-based RL, reaching 60.3% mean on 3DSRBench.
Problem
3D spatial reasoning remains hard for open-source and proprietary LVLMs; prior data-driven fine-tuning performs implicit reasoning and fails to generalize to novel question types requiring compositional 3D computation.
Approach
Use explicit 3D representations (object 3D locations and orientations in a ground-aligned calibrated camera space) as an interface across perception, computation, and reasoning stages, trained by SFT on pseudo-annotated 3D CoT data and then RL with rule-based accuracy and format rewards (KL removed) to improve generalization.
Key results
- SpatialReasoner reaches 60.3% mean accuracy on 3DSRBench, versus 49.8% for Gemini 2.0 Flash and 48.4% for Qwen2.5-VL-7B.
- Trained only on simpler question types, the RL-only SpatialReasoner-Zero gets 46.6% on novel multi-object questions versus 40.0% for SFT-only.
- On CV-Bench-3D distance questions, SpatialReasoner drops to 73.3% from Qwen2.5-VL-7B's 83.2%, which the authors attribute to 2D shortcuts in the benchmark.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add explicit 3D representations (ablation) | SpatialReasoner-SFT · 3DSRBench | accuracy (Mean): 51.9 -> 58.3 | Helped |
| add a second SFT stage on high-quality SR-QA data instead of switching from SFT to RL (ablation) | SpatialReasoner-SFT · 3DSRBench | accuracy (Mean): 58.3 -> 54.7 | Hurt |
| add an RL stage after SFT (ablation) | SpatialReasoner-SFT · 3DSRBench | accuracy (Mean): 58.3 -> 60.3 | Helped |
| add KL divergence regularization to RL (ablation) | SpatialReasoner-Zero · 3DSRBench | accuracy (Mean): 54.0 -> 52.4 | Hurt |
| add 3D-aware process rewards (ablation) | SpatialReasoner-Zero · 3DSRBench | accuracy (Mean): 54.0% to 54.6% | Helped |
| use RL instead of SFT when multi-object training examples are withheld (ablation) | SpatialReasoner · 3DSRBench, multi-object examples withheld during training | accuracy (Multi-Object): 40.0% vs. 46.6% | Helped |
| add RL after SFT when multi-object examples are withheld (ablation) | SpatialReasoner-SFT · 3DSRBench, multi-object examples withheld during training | accuracy (Multi-Object): 40.0% -> 43.4% | Helped |
| add RL after SFT (ablation) | SpatialReasoner-SFT · 3DSRBench | accuracy (Mean): 58.3 -> 60.3 | Helped |
| fine-tune with explicit 3D spatial reasoning instead of the unmodified Qwen2.5-VL-7B-Instruct baseline | SpatialReasoner · 3DSRBench | accuracy (Mean): 52.0 -> 60.3 | Helped |
| fine-tune with explicit 3D spatial reasoning | SpatialReasoner · CVBench-3D | accuracy (Depth): 82.5 -> 87.3 | Helped |
| use explicit 3D reasoning rather than the baseline model's learned approach | SpatialReasoner · CVBench-3D | accuracy (Distance): 83.2 -> 73.3 | Hurt |
| fine-tune with explicit 3D spatial reasoning | SpatialReasoner · GQA | accuracy (Mean): 58.8% to 61.8% | Helped |
Surprising findings
- A simple 2D-center heuristic reaches 80.2% on CV-Bench-3D distance questions, suggesting that baseline models exploit 2D shortcuts.
- Repeated SFT (SFT+SFT) degrades performance from 58.3% to 54.7%, while SFT followed by RL improves it to 60.3%.
Limitations
- RL alone (SpatialReasoner-Zero) fails to produce high-quality reasoning processes, so SFT warm-up data remains necessary.
- The model always performs explicit multi-step 3D reasoning regardless of question difficulty, which is inefficient.
- Training data relies on noisy 3D pseudo-annotations; RL slightly hurts 3D perception and reasoning accuracy because rewards ignore intermediate 3D representations.
Experimental setup
Datasets: OpenImages (pseudo-annotated) LLaVA instruction data 3DSRBench CV-Bench GQA
Benchmarks: 3DSRBench CV-Bench-3D GQA
Metrics: accuracy mean error (location/distance) angle accuracy at pi/6 threshold
Baselines: Qwen2.5-VL-7B-Instruct Qwen2.5-VL-72B-Instruct LLaVA-Next-8B Cambrian-1-8B SpatialRGPT SpatialLLM SpaceLLaVA SpatialBot GPT-4o Gemini 2.0 Flash Claude 3.5 Sonnet QwenVLMax
Builds on: GRPO Qwen2.5-VL SpatialRGPT SpatialLLM
Closest papers by meaning
- SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.93)
- Spatial Understanding from Videos: Structured Prompts Meet Simulation Data (NeurIPS 2025 · similarity 0.92)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.92)
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.92)
- SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.92)
- Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views (CVPR 2026 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.