Research map / Multi-agent systems and games
Cooperative multi-agent RL: research map
635 papers on Cooperative multi-agent RL in Multi-agent systems and games, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 2 clusters and 8 approaches. The busiest year so far is 2026.
What it is
Teams of agents learning to cooperate with reinforcement learning, as in robot teams, games or traffic control. A central difficulty is giving each agent credit for the team's reward; value factorisation and centralised training with decentralised execution are standard tools.
Terms you will see on this page:
- MARL
- multi-agent reinforcement learning.
- CTDE
- centralised training, decentralised execution: train with shared information, act independently.
- Value factorization
- splitting a team value into per-agent parts.
Start reading with the most cited papers here:
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning (NeurIPS 2016 · 1,057 citations)
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (NeurIPS 2017 · 1,002 citations)
Within Multi-agent systems and games, its share shrank from 22.4% in 2023–24 to 11.9% in 2025–26 (141 → 228 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Cooperative multi-agent RL in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- MARL benchmarks and grouping rising: 16% → 27% of the topic (22 → 62 papers)
- Cooperative MARL algorithms rising: 28% → 31% of the topic (40 → 71 papers)
- Offline and networked MARL rising: 11% → 13% of the topic (16 → 30 papers)
- Decentralized actor-critic MARL falling: 11% → 7% of the topic (16 → 15 papers)
- Value factorization falling: 7% → 3% of the topic (10 → 6 papers)
Papers from 2025–2026 that are already cited most:
- Exploring the Efficacy of Multi-Agent Reinforcement Learning for Autonomous Cyber Defence: A CAGE Challenge 4 Perspective (AAAI 2025 · 15 citations)
- Scalable Decentralized Algorithms for Online Personalized Mean Estimation (AAAI 2025 · 5 citations)
Approaches and key papers
Cooperative MARL · 390 papers
Keywords: reinforcement marl, communication, marl algorithms
Approaches in this cluster:
- Cooperative MARL algorithms (154 papers)
Proposes parameter-sharing, primal-dual, and benchmarked algorithms for cooperative multi-agent reinforcement learning. - MARL benchmarks and grouping (139 papers)
Introduces benchmarks and agent grouping or experience-sharing schemes for cooperative MARL. - Learned inter-agent communication (61 papers)
Learns when, what, and how to communicate among agents under bandwidth and delay limits. - Intrinsic-reward exploration for MARL (36 papers)
Drives multi-agent exploration with individual and collective intrinsic rewards and curiosity.
Most cited and most cited since 2024:
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning (NeurIPS 2016 · 1,057 citations)
- The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games (NeurIPS 2022 · 597 citations)
- Carbon Footprint Reduction for Sustainable Data Centers in Real-Time (AAAI 2024 · 25 citations)
- T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration (AAAI 2024 · 18 citations)
Decentralized coordination · 245 papers
Keywords: decentralized, value, joint
Approaches in this cluster:
- Decentralized actor-critic MARL (79 papers)
Trains decentralized or centralized-critic policy-gradient agents with limited communication. - Offline and networked MARL (71 papers)
Learns multi-agent policies from fixed datasets or with networked, scalable value decomposition. - Coordination graphs (53 papers)
Models agent interactions with sparse coordination graphs and search-based planning. - Value factorization (42 papers)
Decomposes joint Q-functions into per-agent values with theory and improved factorization schemes.
Most cited and most cited since 2024:
- Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (NeurIPS 2017 · 1,002 citations)
- Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning (ICML 2017 · 420 citations)
- Intrinsic Action Tendency Consistency for Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · 9 citations)
- STAS: Spatial-Temporal Return Decomposition for Solving Sparse Rewards Problems in Multi-agent Reinforcement Learning (AAAI 2024 · 9 citations)
Related topics in Multi-agent systems and games
- LLM agents (1,294)
- Games and equilibria (537)
- Human-AI and goal-conditioned (1,320)
