Research map / Vision-language and multimodal / Robot action and navigation
SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
Nikolay Nikolov, Giuliano Albanese, Sombit Dey, Aleksandar Yanev, Luc Van Gool, Jan-Nico Zaech, Danda Pani Paudel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 3D pretraining with object-level 3D tasks and MoGe (SPEAR-VLM) to the VLA (ablation) | pi0 VLA on Bridge V2 sink subset · SIMPLER WidowX | avg success rate: 20.8% -> 35.4% | Improved |
| add object-level 3D prompts to the 3D pretraining tasks (vs random-pixel points only) (ablation) | SPEAR-VLM · SIMPLER WidowX | avg success rate: 20.8% -> 35.4% | Improved |
| add MoGe depth encoder to object-level 3D pretraining (ablation) | SPEAR-VLM · SIMPLER WidowX | avg success rate: 26.0% -> 35.4% | Improved |
| object-level 3D tasks without MoGe, vs plain PaliGemma (ablation) | PaliGemma VLA · SIMPLER WidowX | avg success rate: 20.8% -> 26.0% | Improved |
| train SigLIP and MoGe during VLM pretraining (vs frozen) (ablation) | SPEAR-VLM · SIMPLER WidowX | avg success rate: 29.1% -> 35.4% | Improved |
| freeze MoGe during VLA training (vs training it) (ablation) | SPEAR-VLM · SIMPLER WidowX | avg success rate: 18.8% -> 35.4% | Improved |
| initialize pi0 from SPEAR-VLM instead of PaliGemma (ablation) | pi0 trained on DROID · Franka real world (3 tasks) | avg task progress: 34% -> 46% | Improved |
| initialize pi0 from SPEAR-VLM instead of PaliGemma (ablation) | pi0 trained on DROID · Franka Marker in Cup (Dist) | task progress: 67% -> 43% | Hurt |
| SPEAR-1 vs SpatialVLA and OpenVLA in simulation | SPEAR-1 (Bridge) · SIMPLER WidowX | avg success rate: 42.7% -> 57.3% | Improved |
Closest papers by meaning
- 3D-VLA: A 3D Vision-Language-Action Generative World Model (ICML 2024 · similarity 0.90)
- Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.90)
- D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigation (CVPR 2026 · similarity 0.90)
- Robotic Visual Instruction (CVPR 2025 · similarity 0.90)
- Abstract 3D Perception for Spatial Intelligence in Vision-Language Models (CVPR 2026 · similarity 0.90)
- An Embodied Generalist Agent in 3D World (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
