Research map / Vision-language and multimodal / Multimodal LLMs
Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spatial-SSRL, a tool-free self-supervised RLVR paradigm that builds five verifiable pretext tasks from RGB/RGB-D images to improve spatial understanding in Qwen2.5-VL and Qwen3-VL models.
Problem
Spatial understanding is weak in LVLMs, and existing SFT and RLVR pipelines for it rely on costly annotations, specialized tools, or constrained environments that limit scale.
Approach
Repurpose self-supervised pretext tasks (patch reordering, flipped patch recognition, cropped patch inpainting, regional depth ordering, relative 3D position prediction) as verifiable reward functions for GRPO, so ground-truth comes deterministically from image structure and depth with no human or LVLM labels.
Key results
- Spatial-SSRL-3B averages 50.54 vs 45.91 for Qwen2.5-VL-3B across seven spatial benchmarks (+4.63), with +12.37 on Spatial457.
- Spatial-SSRL-7B averages 56.58 vs 52.69 for Qwen2.5-VL-7B (+3.89), with +8.67 on Spatial457.
- Spatial-SSRL-4B (Qwen3-VL-4B) improves spatial average from 60.14 to 61.43 and general VQA average from 69.10 to 70.28.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Self-supervised RLVR training on five pretext tasks (Spatial-SSRL) applied to Qwen2.5-VL-3B | Qwen2.5-VL-3B · 7 spatial benchmarks average | average accuracy: +4.63 | Helped |
| Self-supervised RLVR training on five pretext tasks applied to Qwen2.5-VL-7B | Qwen2.5-VL-7B · 7 spatial benchmarks average | average accuracy: +3.89 | Helped |
| Self-supervised RLVR training applied to Qwen3-VL-4B | Qwen3-VL-4B · 7 spatial benchmarks average | average accuracy: +1.29 | Helped |
| Self-supervised RLVR training applied to Qwen2.5-VL-3B | Qwen2.5-VL-3B · Spatial457 | accuracy: +12.37 | Helped |
| Self-supervised RLVR training applied to Qwen2.5-VL-3B | Qwen2.5-VL-3B · VSI-Bench (video) | accuracy: +5.65 | Helped |
| Self-supervised RLVR training applied to Qwen2.5-VL-7B | Qwen2.5-VL-7B · General VQA (Gnr-VQA) | average accuracy: +0.57 | Helped |
| Self-supervised RLVR training applied to Qwen2.5-VL-3B | Qwen2.5-VL-3B · General VQA (Gnr-VQA) | average accuracy: +2.02 | Helped |
| Self-supervised RLVR training applied to Qwen2.5-VL-7B | Qwen2.5-VL-7B · OCR and chart understanding average | average accuracy: +1.22 | Helped |
| Self-supervised RLVR training applied to Qwen2.5-VL-3B | Qwen2.5-VL-3B · OCR and chart understanding average | average accuracy: +0.12 | No clear effect |
| Reasoning prompting on baseline Qwen2.5-VL-7B (no training) | Qwen2.5-VL-7B · What'sUp | accuracy: -16.34 | Hurt |
| Depth-free shuffled patch reordering only (7B) (ablation) | Qwen2.5-VL-7B · Spatial457-2D subset | accuracy: +4.78 | Helped |
| Depth-based regional depth ordering only (7B) (ablation) | Qwen2.5-VL-7B · 3DSRBench-Height subset | accuracy: +10.87 | Helped |
Surprising findings
- Reasoning prompting degrades Qwen2.5-VL-7B on What'sUp (86.95 to 70.61) and VSI-Bench (38.08 to 32.69) and Qwen2.5-VL-3B on average.
- Training on static images only improves video spatial benchmark VSI-Bench by +5.65 for the 3B model.
Limitations
- Video spatial reasoning is only transferred cross-modally from images; no video-native pretext tasks.
- Depth-based tasks depend on existing depth maps from RGB-D datasets (DIODE, MegaDepth), and the relative position task uses fixed heuristic thresholds.
- Gains on some benchmarks are small (e.g., ViewSpatial +1.24 for 3B, +0.98 for 7B) and some general benchmark differences are within evaluation variance.
Experimental setup
Datasets: COCO DIODE MegaDepth
Benchmarks: Spatial457 3DSRBench SpatialEval QSpatial-plus What'sUp ViewSpatial VSI-Bench MMBench-v1.1 BLINK HallusionBench RealWorldQA OCRBench
Metrics: accuracy mean relative accuracy
Baselines: Qwen2.5-VL-3B Qwen2.5-VL-7B SpatialLadder-3B SpaceR-7B Qwen3-VL-4B
Builds on: GRPO DeepSeek-R1 Visual-RFT Jigsaw-R1 Visual Jigsaw
Closest papers by meaning
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.91)
- SpatialCLIP: Learning 3D-aware Image Representations from Spatially Discriminative Language (CVPR 2025 · similarity 0.90)
- SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning (NeurIPS 2025 · similarity 0.90)
- From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.90)
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.90)
- SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.