Research map / Multi-agent systems and games / Cooperative multi-agent RL
Coordination graphs
Models agent interactions with sparse coordination graphs and search-based planning.
50 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Cooperative multi-agent RL, keyword cluster “decentralized · value · joint”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
10 papers in 2023–24 and 16 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 7.1% to 7.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Solving Transition-Independent Multi-Agent MDPs with Sparse Interactions (AAAI 2016 · 18 citations)
- 2017 · Bounding the Probability of Resource Constraint Violations in Multi-Agent MDPs (AAAI 2017 · 12 citations)
- 2018 · Learning to Coordinate with Coordination Graphs in Repeated Single-Stage Multi-Agent Decision Problems (ICML 2018 · 19 citations)
- 2019 · Successor Features Based Multi-Agent RL for Event-Based Decentralized MDPs (AAAI 2019 · 4 citations)
- 2020 · Deep Coordination Graphs (ICML 2020 · 42 citations)
- 2021 · Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning (ICML 2021 · 15 citations)
- 2022 · Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems (AAAI 2022 · 20 citations)
- 2023 · Models as Agents: Optimizing Multi-Step Predictions of Interactive Local Models in Model-Based Multi-Agent Reinforcement Learning (AAAI 2023 · 13 citations)
- 2024 · Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies (ICML 2024 · 4 citations)
- 2025 · Gradient-Guided Credit Assignment and Joint Optimization for Dependency-Aware Spatial Crowdsourcing (AAAI 2025 · 3 citations)
- 2026 · Achieving Equilibrium Under Utility Heterogeneity: An Agent-Attention Framework for Multi-Agent Multi-Objective Reinforcement Learning (AAAI 2026 · 1 citation)
Earliest papers here
- Solving Transition-Independent Multi-Agent MDPs with Sparse Interactions (AAAI 2016 · 18 citations)
- Multi-Variable Agents Decomposition for DCOPs (AAAI 2016 · 11 citations)
- Bounding the Probability of Resource Constraint Violations in Multi-Agent MDPs (AAAI 2017 · 12 citations)
What has been tried
38 papers here report 346 measured changes (a component added, removed or swapped, compared with a baseline): helped 66%, no clear effect 9%, hurt 19%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add meta-gradient guidance from the reward-shaping policy to the sparse-reward policy | Chengdu · average revenue vs training steps | 6×105 | Helped | Gradient-Guided Credit Assignment and Joint Optimization for Dependency-Aware Spatial Crowdsourcing (AAAI 2025 · 3 citations) |
| add offline weighted particle filtering to FS-POMCP | FireFightingGraph, Multi-Agent RockSample, CaptureTarget · return | Helped | Factored Online Planning in Many-Agent POMDPs (AAAI 2024 · 3 citations) | |
| Remove Dynamic Node Generation (DNG) | MatGame, medium and hard scenarios · reward | Hurt | MALinZero: Efficient Low-Dimensional Search for Mastering Complex Multi-Agent Planning (NeurIPS 2025 · 1 citation) | |
| Use trajectory-class-aware coverage indexing J(t,k) instead of timestep-only J(t) | SurComb4 · mean return | Helped | Trajectory-Class-Aware Multi-Agent Reinforcement Learning (ICLR 2025 · 0 citations) | |
| Enforce non-negative pairwise utility instead of allowing negative pairwise utility | Wolfpack and LBF; maximum of 5 and 9 agents · performance | Helped | Open Ad Hoc Teamwork with Cooperative Game Theory (ICML 2024 · 0 citations) | |
| replace the TrXL actor with a GTrXL actor | Nashville, 26 responders · average response time | 5 seconds higher than TrXL | Hurt | Multi-Agent Reinforcement Learning with Hierarchical Coordination for Emergency Responder Stationing (ICML 2024 · 0 citations) |
| replace Max-Plus with Var-El for action selection | SysAdmin ring · average return | Mixed | Scalable Safe Policy Improvement for Factored Multi-Agent MDPs (ICML 2024) |
What reviewers will expect
Datasets and baselines named in the 9 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- QMIX · 2 of 9 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 26 papers; from OpenAlex and the papers' affiliations).
- Sun Yat-sen University · 2 papers
- Peng Cheng Laboratory · 2 papers
- Huawei · 2 papers
- Eindhoven University of Technology · 2 papers
- Carnegie Mellon University · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Decentralized actor-critic MARL (77 papers)
- Offline and networked MARL (65 papers)
- Value factorization (39 papers)
Elsewhere in the topic
- Cooperative MARL algorithms (149 papers)
- MARL benchmarks and grouping (121 papers)
- Learned inter-agent communication (58 papers)
- Intrinsic-reward exploration for MARL (34 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).