Research map / Probabilistic and scientific modeling / Molecules and proteins
Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeerDrive, an end-to-end driving framework that predicts future BEV scene representations with a transformer world model and iteratively couples them with trajectory planning, reaching state-of-the-art on NAVSIM and nuScenes.
Problem
End-to-end autonomous driving typically uses a one-shot paradigm that relies on the current scene and underestimates how scene dynamics evolve and how the ego vehicle's future actions influence the scene.
Approach
Jointly model future BEV scene evolution and ego trajectory planning in a closed loop: a BEV world model predicts future BEV features that are injected into a decoupled, future-aware planner (fused via motion-aware layer norm), and the planner's refined ego feature is fed back to update the future prediction over N iterations.
Key results
- Achieves PDMS 88.9 on NAVSIM navtest with ResNet34 backbone, versus 88.6 for Hydra-NeXt and 88.1 for DiffusionDrive; 90.7 with V2-99 backbone versus GoalFlow's 90.3.
- On nuScenes validation, average L2 0.43 m and average collision rate 0.06%, improving on SparseDrive (0.61 m, 0.08%).
- Ablation: removing both future-aware planning and iterative modeling drops PDMS from 88.9 to 87.1; two iterations give the best trade-off.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SeerDrive instead of the strongest listed ResNet34 baseline, Hydra-NeXt | NAVSIM navtest planning · NAVSIM navtest | PDMS: 88.6 -> 88.9 | Helped |
| Use SeerDrive with the V2-99 backbone instead of GoalFlow† | NAVSIM navtest planning · NAVSIM navtest | PDMS: 90.3 -> 90.7 | Helped |
| Use SeerDrive instead of the best listed baseline for average L2 | nuScenes planning · nuScenes validation | L2 Avg.: 0.46 -> 0.43 | Helped |
| Use SeerDrive instead of SparseDrive | nuScenes planning · nuScenes validation | Collision Rate Avg. (%): 0.08 -> 0.06 | Helped |
| Add future-aware planning and iterative scene modeling (ablation) | SeerDrive · NAVSIM navtest | PDMS: 87.1 -> 88.9 | Helped |
| Add future-aware planning while retaining iterative modeling (ablation) | SeerDrive · NAVSIM navtest | PDMS: 87.9 -> 88.9 | Helped |
| Add iterative scene modeling and vehicle planning (ablation) | SeerDrive · NAVSIM navtest | PDMS: 88.1 -> 88.9 | Helped |
| Add future BEV injection (ablation) | SeerDrive · NAVSIM navtest | PDMS: 87.9 -> 88.9 | Helped |
| Decouple processing of current and future BEV features (ablation) | SeerDrive · NAVSIM navtest | PDMS: 87.3 -> 88.9 | Helped |
| Use MLN to combine current and future ego features instead of concatenation or addition (ablation) | SeerDrive · NAVSIM navtest | PDMS: 88.3 -> 88.9; 88.5 -> 88.9 | Helped |
| Use 2 iterations instead of 1 or 3 (ablation) | SeerDrive · NAVSIM navtest | PDMS: 88.1 -> 88.9; 88.7 -> 88.9 | Mixed |
| Predict only the final future BEV instead of multiple intermediate BEVs (ablation) | SeerDrive · NAVSIM navtest | PDMS: 88.8 -> 88.9; 88.9 -> 88.9 | No clear effect |
Surprising findings
- Predicting future BEV at intermediate steps (1s-2s-3s-4s) gives no significant gain over predicting only the final step.
Limitations
- BEV world model is a task-specific transformer and lacks the generalization of foundation-model world models.
- Failure cases show trajectory selection errors and inability to infer correct driving intention without high-level commands.
- nuScenes is relatively simple and its metrics are imperfect, as noted by the authors.
Experimental setup
Datasets: NAVSIM nuScenes
Benchmarks: NAVSIM navtest nuScenes validation Bench2Drive
Metrics: PDMS NC DAC TTC Comfort EP L2 error collision rate driving score success rate
Baselines: VADv2 Hydra-MDP UniAD LTF PARA-Drive TransFuser LAW DRAMA Hydra-MDP++ DiffusionDrive WoTE Hydra-NeXt
Builds on: TransFuser DiffusionDrive VAD UniAD
Closest papers by meaning
- FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving (NeurIPS 2025 · similarity 0.91)
- Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving (AAAI 2025 · similarity 0.91)
- DriveFuture: Future-Aware Latent World Models for Autonomous Driving (NeurIPS 2026 · similarity 0.90)
- Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving (CVPR 2024 · similarity 0.89)
- Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving (NeurIPS 2026 · similarity 0.89)
- COME: Adding Scene-Centric Forecasting Control to Occupancy World Model (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.