Research map / Vision-language and multimodal / Robot action and navigation
DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DriveDreamer-Policy, a unified driving world-action model where an LLM (Qwen3-VL-2B) feeds query embeddings to three generative experts (pixel-space depth, latent video, action diffusion), achieving 89.2 PDMS on Navsim v1 and 88.7 EPDMS on Navsim v2.
Problem
Existing driving world-action models model 2D appearance or latent representations with limited geometric grounding, which limits how well imagined futures inform planning.
Approach
Generate an explicit depth map alongside future video and actions from a shared LLM interface, using a causal depth→video→action query-group attention mask so geometry scaffolds both video imagination and planning in a single pass.
Key results
- Achieves 89.2 PDMS on Navsim v1 navtest, the highest among compared methods (previous best listed 88.4 DriveVLA-W0).
- Achieves 88.7 EPDMS on Navsim v2 navtest, reported as +2.6 over the previous method.
- Video FVD of 53.59 versus 85.95 for PWM (single-view front evaluation); depth AbsRel 8.1 versus 9.3 for fine-tuned PPD.
Surprising findings
- Depth-only and video-only world learning each give smaller planning gains than joint depth+video learning, which reaches the best PDMS.
Limitations
- Video evaluation compared only against PWM on single front view, limiting comparability.
- Depth and video components are evaluated against limited baselines; no widely adopted Navsim depth benchmark exists.
- Latency claims are not quantified in the provided text.
Experimental setup
Datasets: Navsim navtrain Depth Anything 3 depth labels
Benchmarks: Navsim v1 navtest Navsim v2 navtest
Metrics: PDMS EPDMS FVD LPIPS PSNR AbsRel delta accuracy
Baselines: TransFuser UniAD PARA-Drive DiffusionDrive AutoVLA ReCogDrive DriveVLA-W0 LAW DrivingGPT WoTE Epona FSDrive
Builds on: flow matching Wan-2.1 PPD Qwen3-VL Epona PWM DriveVLA-W0
Closest papers by meaning
- 3D-VLA: A 3D Vision-Language-Action Generative World Model (ICML 2024 · similarity 0.90)
- From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction (NeurIPS 2025 · similarity 0.89)
- Learning 3D Persistent Embodied World Models (NeurIPS 2025 · similarity 0.89)
- World–Value–Action Model: Implicit Planning for Vision–Language–Action Systems (NeurIPS 2026 · similarity 0.89)
- EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields (NeurIPS 2026 · similarity 0.88)
- Navigation World Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.