Research map / 3D vision / 3D generation
DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DrivingSphere, a generative closed-loop driving simulator that builds a 4D occupancy world (OccDreamer for static backgrounds, actor bank for dynamic objects) and renders multi-view videos with a VideoDreamer diffusion model conditioned on occupancy.
Problem
Autonomous driving evaluation needs simulators that provide both realistic multi-view sensor data and closed-loop feedback; existing open-loop generative methods lack feedback, while closed-loop simulators lack real-world-like visual outputs.
Approach
Compose a 4D occupancy driving world (occupancy diffusion backgrounds with BEV and text conditioning, plus an actor bank driven by a traffic simulator) and render it into multi-view video via dual-path occupancy conditioning and ID-aware actor encoding, enabling closed-loop interaction with end-to-end driving agents.
Key results
- VideoDreamer achieves FVD 103.42 versus 218.12 for MagicDrive and 185.32 for DriveArena on nuScenes validation, with 3D detection mAP 21.45.
- OccDreamer reduces FID to 274 and MMD to 0.082 compared to SemCity's 634 and 0.251.
- In closed-loop UniAD evaluation on 100 routes, route completion rises to 0.1170 from DriveArena's 0.0641.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use occupancy-based DrivingSphere video generation instead of MagicDrive | nuScenes video generation evaluated with UniAD · nuScenes val | FVD: 218.12 -> 103.42 | Helped |
| Use occupancy-based DrivingSphere video generation instead of DriveArena | nuScenes video generation evaluated with UniAD · nuScenes val | 3D object detection mAP: 30.03 -> 34.16 | Helped |
| Use occupancy-based DrivingSphere video generation instead of DriveArena | nuScenes video generation evaluated with UniAD · nuScenes val | NDS: 26.14 -> 27.99 | Helped |
| Use DrivingSphere occupancy generation instead of SemCity | nuScenes occupancy generation · nuScenes occupancy generation | FID: 634 -> 274 | Helped |
| Use DrivingSphere occupancy generation instead of SemCity | nuScenes occupancy generation · nuScenes occupancy generation | MMD: 0.251 -> 0.082 | Helped |
| Use DrivingSphere as the open-loop simulator instead of DriveArena* | UniAD · nuScenes open-loop, 100 predefined trajectories | PDMS: 0.698 -> 0.742 | Helped |
| Use DrivingSphere as the closed-loop simulator instead of DriveArena | UniAD · nuScenes closed-loop, 100 predefined routes | PDMS: 0.6901 -> 0.7281 | Helped |
| Use DrivingSphere as the closed-loop simulator instead of DriveArena | UniAD · nuScenes closed-loop, 100 predefined routes | RC: 0.0641 -> 0.1170 | Helped |
Limitations
- Closed-loop evaluation uses only 100 pre-defined routes and low route completion (about 11.7%), indicating limited scale of testing.
- Environment agent behaviors are limited; the authors say dynamic agent behavior needs enhancement.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes NAVSIM
Metrics: FVD FID MMD mAP NDS L2 PDMS NC DAC TTC
Baselines: MagicDrive Panacea DriveArena SemCity CARLA
Builds on: OpenSora ST-DiT MagicDrive PixArt-delta ControlNet UniAD OccWorld
Closest papers by meaning
- DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation (CVPR 2025 · similarity 0.92)
- Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving (AAAI 2025 · similarity 0.89)
- ReSim: Reliable World Simulation for Autonomous Driving (NeurIPS 2025 · similarity 0.87)
- NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking (NeurIPS 2024 · similarity 0.87)
- Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution (NeurIPS 2025 · similarity 0.87)
- Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.