Research map / Multi-agent systems and games / Cooperative multi-agent RL
Decentralized actor-critic MARL
Trains decentralized or centralized-critic policy-gradient agents with limited communication.
77 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Cooperative multi-agent RL, keyword cluster “decentralized · value · joint”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
16 papers in 2023–24 and 15 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 11.3% to 6.6%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · A primal-dual method for conic constrained distributed optimization problems (NeurIPS 2016 · 4 citations)
- 2017 · Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (NeurIPS 2017 · 1,002 citations)
- 2018 · Counterfactual Multi-Agent Policy Gradients (AAAI 2018 · 1,757 citations)
- 2019 · Actor-Attention-Critic for Multi-Agent Reinforcement Learning (ICML 2019 · 289 citations)
- 2020 · Multi-Agent Actor-Critic with Hierarchical Graph Attention Network (AAAI 2020 · 128 citations)
- 2021 · FACMAC: Factored Multi-Agent Centralised Policy Gradients (NeurIPS 2021 · 104 citations)
- 2022 · A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning (AAAI 2022 · 10 citations)
- 2023 · Consensus Learning for Cooperative Multi-Agent Reinforcement Learning (AAAI 2023 · 28 citations)
- 2024 · Intrinsic Action Tendency Consistency for Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · 9 citations)
- 2025 · Scalable Decentralized Algorithms for Online Personalized Mean Estimation (AAAI 2025 · 5 citations)
- 2026 · Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning (ICML 2026 · 0 citations)
Earliest papers here
- A primal-dual method for conic constrained distributed optimization problems (NeurIPS 2016 · 4 citations)
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (NeurIPS 2017 · 1,002 citations)
- Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability (ICML 2017 · 180 citations)
What has been tried
66 papers here report 524 measured changes (a component added, removed or swapped, compared with a baseline): helped 61%, no clear effect 14%, hurt 20%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| replace the action-model reward with the exploration-based EMC reward | SMAC 6h_vs_8z and 3s_vs_5z · performance | Helped | Intrinsic Action Tendency Consistency for Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · 9 citations) | |
| replace the MLP Policy Modifier with hypernetworks | SMAC-v2 protoss 5v5 · winning rate | 0.68 (0.04) -> 0.60 (0.13) | Hurt | AgentMixer: Multi-Agent Correlated Policy Factorization (AAAI 2025 · 1 citation) |
| vary Leaky ReLU negative slope η from 0 toward 1 (less optimism) | HalfCheetah 6x1 · average episode return | Hurt | Optimistic Multi-Agent Policy Gradient (ICML 2024 · 1 citation) | |
| Replace MisoDICE's learnable mixing with a standard VDN mixer | SMACv2 · average return | 9.2 ± 0.6 vs 8.2 ± 0.5 (Terran 20_vs_20) | Hurt | MisoDICE: Multi-Agent Imitation from Mixed-Quality Demonstrations (NeurIPS 2025 · 0 citations) |
| increase L_enc from 1 toward 3 | GridSim 12×12, group size 6 · reward | Helped | Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning (AAAI 2025) |
What reviewers will expect
Datasets and baselines named in the 12 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- MAPPO · 2 of 12 papers
- MAT · 2 of 12 papers
- MAT-Dec · 2 of 12 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 31 papers; from OpenAlex and the papers' affiliations).
- Chinese Academy of Sciences · 4 papers
- University of Chinese Academy of Sciences · 4 papers
- Nanjing University · 2 papers
- Nanjing University of Posts and Telecommunications · 2 papers
- University of Electronic Science and Technology of China · 2 papers
- Aalto University · 2 papers
- Inria · 2 papers
- Yale University · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Offline and networked MARL (65 papers)
- Coordination graphs (50 papers)
- Value factorization (39 papers)
Elsewhere in the topic
- Cooperative MARL algorithms (149 papers)
- MARL benchmarks and grouping (121 papers)
- Learned inter-agent communication (58 papers)
- Intrinsic-reward exploration for MARL (34 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).