Research map / Vision-language and multimodal / Multimodal LLMs

SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SpaceDrive, a VLM-based end-to-end driving planner that encodes 3D coordinates with a shared sine-cosine positional encoding added to visual tokens, used for text coordinates and ego history, and decoded by a regression head instead of digit tokens.

Problem

VLM-based end-to-end driving planners struggle with fine-grained 3D spatial reasoning and produce inaccurate waypoints because language models generate numbers digit-by-digit and lack 3D-grounded pretraining.

Approach

A universal 3D sine-cosine positional encoding, computed from depth-lifted patch coordinates, is added to visual tokens and also replaces textual coordinate digits as both VLM input and output, with a learnable MLP regression decoder for coordinates and a learnable normalization factor for the PE.

Key results

• On nuScenes open-loop with ego status (SpaceDrive+), average L2 0.32 m, collision 0.23%, intersection 1.27%, reported as SOTA among VLM-based methods.
• On Bench2Drive closed-loop, SpaceDrive+ achieves Driving Score 78.02 and Success Rate 55.11%, second-best among VLM-based methods behind SimLingo (85.07).
• Without ego status (SpaceDrive), average L2 1.80 vs 1.98 for OmniDrive-Q, with ablation showing PE on vision tokens drops avg L2 from 2.51 to 1.88.

Surprising findings

• Adding the 3D PE to vision tokens alone yields most of the gain, while replacing textual coordinates with PE alone gives smaller improvements.
• Using a sine-cosine PE as the decoder or RoPE as the encoder degrades performance, and RoPE causes large instability due to clashing with the VLM's RoPE.

Limitations

• No explicit uncertainty handling, as stated by the authors.
• No exploitation of multi-frame temporal memory.
• Closed-loop result is second-best behind SimLingo, which uses extensive data augmentation.

Experimental setup

Datasets: nuScenes OmniDrive VQA data SimLingo routes (3600 episodes, PDM-lite expert) Bench2Drive

Benchmarks: nuScenes Bench2Drive

Metrics: L2 (m) Collision (%) Intersection (%) Driving Score Success Rate (%)

Baselines: OmniDrive-Q OmniDrive-Q++ ORION DriveVLM EMMA RDA-Driver SimLingo VDRive UniAD VAD-Base

Builds on: OmniDrive ORION SimLingo Qwen2.5-VL UniDepth

Closest papers by meaning

• Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.92)
• SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving (CVPR 2026 · similarity 0.91)
• DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.91)
• LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.90)
• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving (NeurIPS 2025 · similarity 0.90)
• SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models (NeurIPS 2025 · similarity 0.90)