Research map / Image generation / Fast diffusion sampling
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion-ES, which combines evolutionary search over trajectories sampled from an unconditional diffusion model with truncated diffuse-denoise mutations, to optimize black-box driving rewards and LLM-generated language-shaped rewards in nuPlan.
Problem
Reward-gradient guided diffusion requires differentiable rewards fitted on noisy samples, limiting its use for general black-box trajectory optimization in closed-loop driving and instruction following.
Approach
Use a diffusion model to initialize and mutate populations in evolutionary search: elites are mutated by partially noising and then denoising them (truncated diffusion), and scored by a black-box reward on clean samples only, so no noisy-sample reward model is needed.
Key results
- Diffusion-ES achieves a driving score of 92 on nuPlan Val14 reactive track, matching PDM-Closed (92) and outperforming diffusion policy (50), PlanCNN (72), and UrbanDriverOL (65).
- On lane following, Diffusion-ES reduces lane error to 0.61 versus 1.22 for reward-gradient guidance and 2.34 for CEM.
- Diffusion-ES outperforms all baselines on the eight language instruction tasks, though PDM-Closed-Multilane beats it on 4 of 8 tasks (per the text's wording).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Diffusion-ES instead of PDM-Closed | nuPlan Val14 driving · nuPlan Val14 | Driving Score: 92 -> 92 | No clear effect |
| Use Diffusion-ES instead of the Diffusion policy | nuPlan Val14 driving · nuPlan Val14 | Driving Score: 50 -> 92 | Helped |
| Use Diffusion-ES instead of reactive neural policies | nuPlan Val14 driving · nuPlan Val14 | Driving Score: 65 -> 92; 72 -> 92 | Helped |
| Use Diffusion-ES instead of CEM | nuPlan lane following · nuPlan lane following | Lane Error: 2.34 -> 0.61 | Helped |
| Use Diffusion-ES instead of MPPI | nuPlan lane following · nuPlan lane following | Lane Error: 3.29 -> 0.61 | Helped |
| Use Diffusion-ES instead of reward-gradient guidance | nuPlan lane following · nuPlan lane following | Lane Error: 1.22 -> 0.61 | Helped |
| Use Diffusion-ES instead of reward-gradient guidance | nuPlan lane following · nuPlan lane following | Speed Error: 0.96 -> 0.79 | Helped |
| Use Diffusion-ES instead of CEM | nuPlan lane following · nuPlan lane following | Speed Error: 2.05 -> 0.79 | Helped |
| Use Diffusion-ES instead of MPPI | nuPlan lane following · nuPlan lane following | Speed Error: 2.74 -> 0.79 | Helped |
| Condition the diffusion model on scene features (ablation) | Diffusion-ES · Language instruction following tasks | Task success rate | Hurt |
| Reduce diffusion steps to T = 10, population size to M = 32, and search iterations to K = 2 (ablation) | Diffusion-ES · nuPlan Val14 | Driving Score: 92 -> 91 | Hurt |
| Reduce diffusion steps to T = 10, population size to M = 32, and search iterations to K = 2 (ablation) | Diffusion-ES · Inference | Wallclock time (s): 5.85 ± 0.11 -> 0.50 ± 0.01 | Helped |
Surprising findings
- Diffusion-ES outperforms reward-gradient guidance even on a differentiable lane-following objective, which the authors attribute to gradients being uninformative on noisy intermediate trajectories.
- Conditioning the diffusion model on scene context substantially degrades Diffusion-ES performance on instruction following.
Limitations
- Substantial computational overhead from iterative denoising and population search (5.85 s per timestep in default setting).
- Reward assumes other agents move at constant velocity and do not react to the ego vehicle.
- Conditional diffusion variant performs much worse because scene context makes guidance harder.
Experimental setup
Datasets: nuPlan
Benchmarks: nuPlan Val14 (reactive agents) nuPlan instruction following tasks nuPlan lane following
Metrics: driving score task success rate lane error speed error wallclock time
Baselines: UrbanDriverOL PlanCNN IDM PDM-Closed Diffusion Policy PDM-Closed-Multilane CEM MPPI Reward-gradient guidance
Builds on: diffusion models MPPI CEM reward-gradient guidance PDM-Closed
Closest papers by meaning
- DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving (CVPR 2025 · similarity 0.91)
- Diffusion-Based Planning for Autonomous Driving with Flexible Guidance (ICLR 2025 · similarity 0.91)
- DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising (NeurIPS 2026 · similarity 0.90)
- Prior-Guided Diffusion Planning for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Model-based Diffusion for Trajectory Optimization (NeurIPS 2024 · similarity 0.89)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.