Research map / Label-efficient and robust learning / Self-supervised and contrastive

Robust Autonomy Emerges from Self-Play

ICML 2025 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Self-supervision in supervised pipelines · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that a single generalist driving policy trained purely by self-play at massive scale (Gigaflow, 1.6B km) achieves state-of-the-art zero-shot results on CARLA, nuPlan, and Waymax without human data.

Problem

Whether self-play RL, at unprecedented scale, can yield robust and naturalistic autonomous driving policies without human driving data or hand-designed scenarios.

Approach

A batched GPU simulator (Gigaflow) enables self-play RL with a single parameterized policy shared by all traffic participants (vehicles, trucks, cyclists, pedestrians), conditioned on dynamics and reward parameters, trained with PPO plus advantage filtering on minimal rewards.

Key results

• Zero-shot, a single self-play policy surpasses prior state of the art on CARLA, nuPlan, and Waymax without training on any benchmark data.
• Averages over 3 million km (17.5 years of continuous driving) between incidents in long-form simulation.
• Scores 0.62 on the WOSAC realism meta-metric in zero-shot evaluation, outperforming several supervised autoregressive approaches.

Surprising findings

• Policies trained purely in self-play with minimalistic rewards and no human data outperform specialists trained on benchmark-specific datasets.
• Many complex skills (merging, unprotected turns) only emerge after 10^11 to 10^12 training steps.

Limitations

• Entirely in simulation; sim-to-real transfer not demonstrated.
• Perception is abstracted away; sensing and perception not modeled.
• Some benchmark infractions are attributed to benchmark limitations rather than the policy.

Experimental setup

Datasets: synthetic urban maps (8 maps, 136 km of road)

Benchmarks: CARLA nuPlan Waymax (Waymo Open Motion Dataset) Waymo Open Sim Agents Challenge (WOSAC)

Metrics: benchmark driving score collisions per scenario realism meta-metric km between incidents

Baselines: dataset-specific specialist policies on CARLA nuPlan state-of-the-art planners Waymax state-of-the-art policies supervised autoregressive sim-agent models on WOSAC

Builds on: PPO Prioritized Experience Replay Deep Sets Intelligent Driver Model

Closest papers by meaning

• Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.88)
• Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving (NeurIPS 2024 · similarity 0.87)
• Autonomous Driving Research Requires a Community-Driven Data Paradigm (NeurIPS 2026 · similarity 0.86)
• Learning to Drive in New Cities Without Human Demonstrations (NeurIPS 2026 · similarity 0.86)
• ReSim: Reliable World Simulation for Autonomous Driving (NeurIPS 2025 · similarity 0.85)
• Hybrid-Driving: An Autonomous Driving Decision Framework Integrating Large Language Models, Knowledge Graphs and Driving Rules (AAAI 2025 · similarity 0.85)