Research map / Multi-agent systems and games / Cooperative multi-agent RL
GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GPUDrive, a GPU-accelerated, data-driven multi-agent driving simulator built on Madrona that reaches over a million agent steps per second and trains goal-reaching RL agents on Waymo Open Motion Dataset scenarios.
Problem
Multi-agent RL for driving planning needs billions of environment samples, but existing driving simulators are too slow (CPU-bound) or lack multi-agent, sensor, and real-data support.
Approach
A C++/CUDA multi-agent driving simulator built on the Madrona ECS engine, with BVH-based collision checking, polyline decimation, per-world memory allocation, and Python (torch/jax Gymnasium) interfaces, using WOMD logs with goal-reaching rewards.
Key results
- GPUDrive achieves a peak of 2.3 million agent steps per second, 2 to 3 orders of magnitude faster than Nocturne (peak ~15,000 ASPS).
- Solving 10 scenarios to 95% goal-reaching takes under 3 minutes versus about 10 hours in Nocturne, a 200-300x training speedup.
- Agents reach 95% goal-reaching on 1000 training scenes in about 15 hours; per-scene cost drops to about 15 seconds at 1024 scenes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPUDrive instead of Nocturne | multi-agent simulation on WOMD scenarios · Waymo Open Motion Dataset scenarios | ASPS: 15,000 to 2.3 million steps | Helped |
| Use GPUDrive instead of Nocturne | IPPO training · 10 WOMD scenarios | training speed: 200 - 300x | Helped |
| Train GPUDrive RL agents | GPUDrive RL agents · 1000 training scenes | goal-reaching rate: 95% | Helped |
| Use LiDAR observation instead of radial filter observation (ablation) | GPUDrive · Waymo Open Motion Dataset scenarios | simulation speed: approximately 3x | Helped |
| Increase training set from 32 to 1024 scenarios | IPPO in GPUDrive · WOMD training scenarios | average completion time per scenario: 2 minutes per scenario to only 15 seconds per scenario | Helped |
| Apply polyline decimation to reduce road geometry points (ablation) | GPUDrive · simulation performance | step time: 10-15 times | Helped |
Surprising findings
- Amortized time per scene decreases as the number of training scenes grows (sub-linear time-to-completion), with 1024 scenes costing about 15 seconds per additional scene.
- Controllable agent steps per second on randomly sampled WOMD scenes (~200K) is much lower than the peak raw ASPS because most agents are parked or already at goal.
Limitations
- Agents reach roughly 95% goal-reaching and still crash or leave the road, far from human-level performance.
- No lane or map connectivity; collision checking only for convex objects.
- Around 2% of goals are unreachable due to mislabeled road edges in WOMD, capping the goal-reaching rate near 98%.
Experimental setup
Datasets: Waymo Open Motion Dataset
Benchmarks: Waymo Open Motion Dataset scenarios
Metrics: agent steps per second (ASPS) controlled agent steps per second (CASPS) goal-reaching rate time to solve scenes collision rate road-edge crossing
Baselines: Nocturne Waymax Stable Baselines 3 IPPO
Builds on: Madrona Independent PPO Waymax Nocturne PufferLib
Closest papers by meaning
- StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning (ICML 2026 · similarity 0.84)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.84)
- MAPF-GPT: Imitation Learning for Multi-Agent Pathfinding at Scale (AAAI 2025 · similarity 0.83)
- World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning (NeurIPS 2026 · similarity 0.83)
- ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management (ICML 2026 · similarity 0.83)
- Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.