Research map / Multi-agent systems and games / Cooperative multi-agent RL
IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces IntersectionZoo, a data-driven multi-agent contextual RL benchmark for cooperative eco-driving at 16,334 US signalized intersections, and benchmarks PPO, DDPG, MAPPO, and GCRL, finding they fail to generalize.
Problem
Multi-agent reinforcement learning generalization across problem variations in real-world settings lacks standardized, realistic contextual RL (CRL) benchmarks.
Approach
A multi-agent CRL benchmark built on cooperative eco-driving at signalized intersections, with context distributions derived from real city data (OSM, AADT, CitySim, MOVES-based emission models) across 10 US cities, supporting IID, systematicity, and productivity (OOD) evaluations.
Key results
- In-distribution (IID) tests on SLC and Atlanta: all four algorithms (PPO, DDPG, MAPPO, GCRL) show many failure cases with higher emissions and/or lower throughput than the human-driving baseline; DDPG has the best success rate but is still far from successful.
- Systematicity test with procedurally generated intersections: both DDPG and PPO fail to generalize systematically; the human-driving baseline performs better in almost all cases.
- Productivity test (zero-shot transfer from SLC to Atlanta): both DDPG and PPO continue to perform poorly.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPO, DDPG, MAPPO, or GCRL for eco-driving on the same city CMDP | PPO, DDPG, MAPPO, and GCRL · Salt Lake City and Atlanta IID evaluation | emission benefits | Hurt |
| Use DDPG instead of PPO, MAPPO, or GCRL | Multi-agent eco-driving policies · Salt Lake City and Atlanta IID evaluation | success rate | Helped |
| Train PPO or DDPG on sampled context-feature combinations and test on unseen combinations | PPO and DDPG · Procedurally generated intersections, systematicity evaluation | emission benefits | Hurt |
| Transfer a policy trained on Salt Lake City to Atlanta | PPO and DDPG · Atlanta, productivity evaluation | emission benefits | Hurt |
Surprising findings
- Popular multi-agent RL algorithms fail even in in-distribution evaluation on the same CMDP.
- Generalization performance differs across cities for the same algorithm, indicating CMDPs pose different challenges.
Limitations
- Primarily supports continuous control benchmarking; discrete lane-changing control is only provided as an option.
- Scenarios may deviate from real-world counterparts due to data errors and missing data.
- Eco-driving is decomposed into individual intersections, assuming non-saturated traffic flow.
Experimental setup
Datasets: OpenStreetMap AADT traffic data CitySim MOVES databases NOAA climate data USGS road grades
Benchmarks: IntersectionZoo (SLC and Atlanta CMDPs)
Metrics: average CO2 emission benefit relative to human-driving basel intersection throughput change
Baselines: PPO DDPG MAPPO GCRL human-driving baseline (calibrated IDM)
Builds on: SUMO Intelligent Driver Model RLlib MOVES Dec-POMDP
Closest papers by meaning
- Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.89)
- Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.89)
- RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning (CVPR 2026 · similarity 0.87)
- URB - Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles (NeurIPS 2025 · similarity 0.87)
- MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving (NeurIPS 2026 · similarity 0.87)
- Learning to Drive in New Cities Without Human Demonstrations (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.