Research map / Multi-agent systems and games / Cooperative multi-agent RL
MARL benchmarks and grouping
Introduces benchmarks and agent grouping or experience-sharing schemes for cooperative MARL.
121 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Cooperative multi-agent RL, keyword cluster “reinforcement marl · communication · marl algorithms”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
22 papers in 2023–24 and 62 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 15.6% to 27.2%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Global Model Checking on Pushdown Multi-Agent Systems (AAAI 2016 · 7 citations)
- 2018 · Modeling Others using Oneself in Multi-Agent Reinforcement Learning (ICML 2018 · 66 citations)
- 2019 · Collective Online Learning of Gaussian Processes in Massive Multi-Agent Systems (AAAI 2019 · 28 citations)
- 2020 · Google Research Football: A Novel Reinforcement Learning Environment (AAAI 2020 · 274 citations)
- 2021 · PettingZoo: Gym for Multi-Agent Reinforcement Learning (NeurIPS 2021 · 146 citations)
- 2022 · The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (NeurIPS 2022 · 597 citations)
- 2023 · SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning (NeurIPS 2023 · 28 citations)
- 2024 · Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · 25 citations)
- 2025 · Exploring the Efficacy of Multi-Agent Reinforcement Learning for Autonomous Cyber Defence: A CAGE Challenge 4 Perspective (AAAI 2025 · 15 citations)
- 2026 · LLM Collaboration with Multi-Agent Reinforcement Learning (AAAI 2026 · 7 citations)
Earliest papers here
- Global Model Checking on Pushdown Multi-Agent Systems (AAAI 2016 · 7 citations)
- Detection of Plan Deviation in Multi-Agent Systems (AAAI 2016 · 1 citation)
- Modeling Others using Oneself in Multi-Agent Reinforcement Learning (ICML 2018 · 66 citations)
What has been tried
71 papers here report 652 measured changes (a component added, removed or swapped, compared with a baseline): helped 63%, no clear effect 9%, hurt 23%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Combine load shifting, HVAC energy optimization, and battery operation in DC-CFR rather than using LS+EO | Arizona, 1-year simulation · carbon footprint reduction vs ASHRAE (%) | 13.26 ± 0.07 -> 14.36 ± 0.09 | Helped | Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · 25 citations) |
| Add CQR | KuaiRand simulator · WatchTime (s) | 782.5 -> 887.2 | Helped | UNEX-RL: Reinforcing Long-Term Rewards in Multi-Stage Recommender Systems with UNidirectional EXecution (AAAI 2024 · 5 citations) |
| Use collaborative reward sharing with α = 0.8 rather than no collaboration (α = 1.0) | SustainDC · CFP (kgCO2) | 176.3 -> 176.2 | Helped | SustainDC: Benchmarking for Sustainable Data Center Control (NeurIPS 2024 · 5 citations) |
| add the Intent Tracker to COMBO | TDW-Game · success rate | 0.65 -> 1.00; 0.60 -> 1.00 | Helped | COMBO: Compositional World Models for Embodied Multi-Agent Cooperation (ICLR 2025 · 2 citations) |
| Add contrastive role learning | SMAC super-hard maps · test win rate | Helped | Attention-Guided Contrastive Role Representations for Multi-agent Reinforcement Learning (ICLR 2024 · 2 citations) | |
| Use the correct hyperparameter action order instead of the reverse order | Seq-Sigmoid variants · return | Helped | Sequential Multi-Agent Dynamic Algorithm Configuration (NeurIPS 2025 · 1 citation) | |
| Use an appropriate sequence length | Pistonball 6agents · average return | Helped | Acting Beyond Learning: Imagination-Assisted Decision-Making in the Visual-based Multi-Agent Cooperative Scenarios (AAAI 2025 · 0 citations) | |
| Select the initial agent by minimizing Qtot instead of random selection | 8m and MMM · robustness | Mixed | Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learning (ICML 2025 · 0 citations) | |
| replace CB worst-case group prediction with random group selection | Simplified warehouse, 9 groups · Recirculation rate | 0.56\% ± 0.18\% -> 1.56\% ± 1.45\% | Hurt | Distributionally Robust Multi-Agent Reinforcement Learning for Dynamic Chute Mapping (ICML 2025 · 0 citations) |
| Add history input and knowledge distillation to Flash + DR | TAG fixed delay 6 · reward | 150.4±22.5 -> 182.5±32.0 | Helped | Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Observation Delays (NeurIPS 2025 · 0 citations) |
| remove selected force-related and history features | Crystal optimization test sets · mean optimization time | MACS substantially better than feat.7 and feat.8 | Hurt | MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures (NeurIPS 2025 · 0 citations) |
| Add climate event probabilities and occurrences to observations | Default 5-company, 3-investor scenario · final climate risk | Helped | InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma (ICLR 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 29 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Google Research Football · 3 of 29 papers
- SMAC · 2 of 29 papers
- Hanabi · 2 of 29 papers
Baselines
- MAPPO · 6 of 29 papers
- GRPO · 3 of 29 papers
- RANDOM · 2 of 29 papers
- PPO · 2 of 29 papers
- GSPO · 2 of 29 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 84 papers; from OpenAlex and the papers' affiliations).
- University of Oxford · 4 papers
- University of Chinese Academy of Sciences · 3 papers
- Meta · 3 papers
- Nanjing University · 3 papers
- Tsinghua University · 3 papers
- Google DeepMind · 3 papers
- Hewlett Packard Labs @ Hewlett Packard Enterprise · 2 papers
- Cornell University · 2 papers
- Singapore Management University · 2 papers
- University of Texas, Austin · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Cooperative MARL algorithms (149 papers)
- Learned inter-agent communication (58 papers)
- Intrinsic-reward exploration for MARL (34 papers)
Elsewhere in the topic
- Decentralized actor-critic MARL (77 papers)
- Offline and networked MARL (65 papers)
- Coordination graphs (50 papers)
- Value factorization (39 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).