Research map / Multi-agent systems and games / Human-AI and goal-conditioned
SceneDiffuser++: City-Scale Traffic Simulation via a Generative World Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SceneDiffuser++, a unified diffusion-based generative world model trained with a single denoising loss that enables city-scale trip-level traffic simulation with agent spawning/removal, occlusion, and traffic light simulation via autoregressive rollout.
Problem
Existing learned traffic simulators handle only short logged event clips (<10s) with fixed agent sets and ignore dynamic agent generation (spawning/removal), occlusion, and traffic light states, so they cannot simulate trip-level (point A to B) driving realistically.
Approach
Model agent validity/visibility as an extra channel in a sparse scene tensor and jointly denoise agents and traffic lights with a multi-tensor diffusion transformer; handle sparsity via zeroing invalid values, validity-weighted loss masking, and an inference-time soft-clipping step.
Key results
- SceneDiffuser++ as world model with IDM planner reduces the composite JS-divergence to 0.2878 versus 0.4793 for IDM and 0.4797 for SceneDiffuser in 60s rollouts.
- With SceneDiffuser++ as both planner and world model, the composite score is 0.2423 across 60s rollouts, the best among the compared planner/world-model pairs.
- Soft clipping gives a composite of 0.2046 versus 0.2498 for hard clipping and 0.2663 for no clipping in the inference-time ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SceneDiffuser++ as the world model rather than IDM | IDM planner on WOMD-XLMap · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.4793 -> 0.2878 | Helped |
| Use SceneDiffuser++ as the world model rather than SceneDiffuser | IDM planner on WOMD-XLMap · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.4797 -> 0.2878 | Helped |
| Use SceneDiffuser++ as the world model rather than IDM | SceneDiffuser++ planner on WOMD-XLMap · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.4234 -> 0.2423 | Helped |
| Use SceneDiffuser++ as the world model rather than SceneDiffuser | SceneDiffuser++ planner on WOMD-XLMap · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.4389 -> 0.2423 | Helped |
| Use 10 replan steps instead of 40 (ablation) | SceneDiffuser++ planner and world model · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.2423 -> 0.1867 | Helped |
| Use 80 replan steps instead of 40 (ablation) | SceneDiffuser++ planner and world model · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.2423 -> 0.2102 | Hurt |
| Extend rollout from 600 to 3000 steps (ablation) | SceneDiffuser++ planner and world model · WOMD-XLMap | Composite JS divergence: 0.2423 -> 0.2468 | Hurt |
| Use soft clipping instead of hard clipping (ablation) | SceneDiffuser++ inference · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.2498 -> 0.2046 | Helped |
| Use soft clipping instead of hard-validity clipping (ablation) | SceneDiffuser++ inference · WOMD-XLMap, 60s rollout | Composite JS divergence: 0.3185 -> 0.2046 | Helped |
| Add soft clipping rather than using no clipping (ablation) | SceneDiffuser++ inference · WOMD-XLMap, 60s rollout | Collision rate JS divergence: 0.6697 -> 0.4194 | Helped |
Surprising findings
- SceneDiffuser++ has worse offroad and collision rates than IDM and SceneDiffuser as a world model, because it inserts agents regardless of the planner's route, sometimes onto the planner's path.
- SceneDiffuser's agents tend to become static during long rollouts because it must keep all agents within the AV's visible range.
Limitations
- Higher offroad and collision rates than baselines, partly due to agents inserted in parking lots that stay parked and inserted onto planner routes.
- Rollout error accumulates over long horizons, degrading realism metrics beyond 60s.
- Evaluation relies on an augmented WOMD-XLMap that is not released.
Experimental setup
Datasets: Waymo Open Motion Dataset (WOMD) WOMD-XLMap
Benchmarks: WOMD-XLMap
Metrics: Jensen-Shannon divergence composite score collision rate offroad rate average speed traffic light violation rate traffic light transition divergence
Baselines: SceneDiffuser Intelligent Driver Model (IDM)
Builds on: SceneDiffuser amortized diffusion v-prediction diffusion Intelligent Driver Model
Closest papers by meaning
- SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout (NeurIPS 2024 · similarity 0.90)
- Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments (CVPR 2025 · similarity 0.89)
- SceneCrafter: Controllable Multi-View Driving Scene Editing (CVPR 2025 · similarity 0.89)
- X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability (NeurIPS 2025 · similarity 0.88)
- Generating Multimodal Driving Scenes via Next-Scene Prediction (CVPR 2025 · similarity 0.88)
- DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.