Research map / Multi-agent systems and games / Cooperative multi-agent RL
Cooperative MARL algorithms
Proposes parameter-sharing, primal-dual, and benchmarked algorithms for cooperative multi-agent reinforcement learning.
149 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in Cooperative multi-agent RL, keyword cluster “reinforcement marl · communication · marl algorithms”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
40 papers in 2023–24 and 71 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 28.4% to 31.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization (NeurIPS 2018 · 116 citations)
- 2019 · Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient (AAAI 2019 · 293 citations)
- 2020 · Multi-agent Reinforcement Learning for Networked System Control (ICLR 2020 · 69 citations)
- 2021 · Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks (NeurIPS 2021 · 75 citations)
- 2022 · LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning (NeurIPS 2022 · 21 citations)
- 2023 · Effective and Stable Role-Based Multi-Agent Collaboration by Structural Information Principles (AAAI 2023 · 45 citations)
- 2024 · Leveraging Partial Symmetry for Multi-Agent Reinforcement Learning (AAAI 2024 · 15 citations)
- 2025 · POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfinding (ICLR 2025 · 6 citations)
- 2026 · HCPO: Hierarchical Conductor-Based Policy Optimization in Multi-Agent Reinforcement Learning (AAAI 2026 · 2 citations)
Earliest papers here
- Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization (NeurIPS 2018 · 116 citations)
- Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient (AAAI 2019 · 293 citations)
- Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI (ICML 2019 · 28 citations)
What has been tried
98 papers here report 942 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 11%, hurt 19%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add symmetry augmentation and symmetry loss with a fixed coefficient of 0.5, without PSE's adaptive framework | Tasks with increasing noise intensity · performance | Hurt | Leveraging Partial Symmetry for Multi-Agent Reinforcement Learning (AAAI 2024 · 15 citations) | |
| Use uniform-policy intervention sampling instead of sampling actions from the replay buffer | Cooperative Predator Prey (5 agents) · reward | Helped | Situation-Dependent Causal Influence-Based Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · 11 citations) | |
| Add negative knowledge sharing (rather than omit it) | PGM-6ag · team training reward | Helped | Cautiously-Optimistic Knowledge Sharing for Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · 6 citations) | |
| Use relation-weighted feature aggregation instead of sum aggregation during curriculum transfer | 8m vs 10m · test win rate | 0.8422(±0.0811) -> 0.9074(±0.0736) | Helped | PORTAL: Automatic Curricula Generation for Multiagent Reinforcement Learning (AAAI 2024 · 6 citations) |
| Replace novelty-guided updates with fixed multiple updates | 6h_vs_8z · win rate | Hurt | Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learning (AAAI 2025 · 4 citations) | |
| replace self-attention with MLPs | SMAC 3s5z · win rate | Hurt | SrSv: Integrating Sequential Rollouts with Sequential Value Estimation for Multi-agent Reinforcement Learning (AAAI 2025 · 3 citations) | |
| Use dCAE embedding instead of random projection | 3s_vs_5z and 5m_vs_6m · overall win-rate | 0.413 -> 0.514; 0.646 -> 0.677 | Helped | Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learning (ICLR 2024 · 3 citations) |
| learn stochastic policies with the MaxEnt objective instead of deterministic policies with fixed Gaussian exploration noise | Ant-v2 4x2 · reward | Helped | Maximum Entropy Heterogeneous-Agent Reinforcement Learning (ICLR 2024 · 3 citations) | |
| Sample M3FC actions per agent (decentralized execution) instead of using one shared, correlated M3FC action | Beach and other evaluated settings · mean episode return | No clear effect | Major-Minor Mean Field Multi-Agent Reinforcement Learning (ICML 2024 · 3 citations) | |
| use ACL-LFT with cross-agent history sharing disabled | SMACv2 3s5z vs 3s6z · performance | 76.1 ± 2.9 -> 78.9 ± 2.8; 72.6 ± 3.2 -> 78.9 ± 2.8 | Helped | Adaptive Context Length Optimization with Low-Frequency Truncation for Multi-Agent Reinforcement Learning (NeurIPS 2025 · 2 citations) |
| Use EIR-MAPPO with posterior beliefs instead of ex ante robust MAPPO without belief | Toy · average reward under diverse uncertainties | 5.81% | Helped | Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Game (ICLR 2024 · 2 citations) |
| Use a lower soft-update coefficient tau | Multi-Agent Navigation · variance of measured SND | Helped | Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning (ICML 2024 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 37 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- SMAC · 6 of 37 papers
- SMACv2 · 5 of 37 papers
- Google Research Football · 2 of 37 papers
- Multi-Agent MuJoCo · 2 of 37 papers
- SMAX · 2 of 37 papers
- LBF · 2 of 37 papers
- MAMuJoCo · 2 of 37 papers
Baselines
- QMIX · 11 of 37 papers
- MAPPO · 10 of 37 papers
- QPLEX · 6 of 37 papers
- IPPO · 6 of 37 papers
- VDN · 5 of 37 papers
- HAPPO · 4 of 37 papers
- MAT · 3 of 37 papers
- MAMBA · 3 of 37 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 111 papers; from OpenAlex and the papers' affiliations).
- Peking University · 7 papers
- Beihang University · 6 papers
- Tsinghua University · 6 papers
- Chinese Academy of Sciences · 6 papers
- InstaDeep · 4 papers
- Hong Kong University of Science and Technology · 4 papers
- Chinese academy of science · 4 papers
- Shanghai Jiao Tong University · 3 papers
- Nanjing University · 3 papers
- Carnegie Mellon University · 3 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- MARL benchmarks and grouping (121 papers)
- Learned inter-agent communication (58 papers)
- Intrinsic-reward exploration for MARL (34 papers)
Elsewhere in the topic
- Decentralized actor-critic MARL (77 papers)
- Offline and networked MARL (65 papers)
- Coordination graphs (50 papers)
- Value factorization (39 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).