Research map / Multi-agent systems and games / Cooperative multi-agent RL
The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PPO with centralized value function (MAPPO) instead of QMix | cooperative MARL · GRF academy scenarios | success rate: 8.12 -> 88.03 (3v.1) | Helped |
| MAPPO instead of QMix | cooperative MARL · SMAC super hard 27m_vs_30m | win rate: 39.1(9.8) -> 93.8(2.4) | Helped |
| centralized critic (MAPPO) instead of decentralized (IPPO) (ablation) | PPO · Hanabi 5 players | avg score: 20.75 -> 23.04 | Helped |
| parameter sharing across agents (ablation) | MAPPO · SMAC 6h_vs_8z | win rate: 11.4(2.5) -> 88.3(3.7) | Helped |
| add value normalization (ablation) | MAPPO · MPE Spread, SMAC | return / win rate | Helped |
| agent-specific global state (AS/FP) instead of concatenated local obs (CL) (ablation) | MAPPO critic · SMAC | win rate | Helped |
| increase training epochs to 15 (ablation) | MAPPO · SMAC hard maps | win rate | Hurt |
| split into 4 mini-batches instead of 1 (ablation) | MAPPO · SMAC | win rate | Hurt |
| large clip epsilon (0.2-0.5) instead of 0.05 (ablation) | MAPPO · SMAC | win rate | Hurt |
| increase batch size beyond critical point (ablation) | MAPPO · SMAC, GRF | sample efficiency | Mixed |
| add death masking with agent ID (ablation) | MAPPO · SMAC | win rate | Helped |
Closest papers by meaning
- Learning Diverse Risk Preferences in Population-Based Self-Play (AAAI 2024 · similarity 0.88)
- PPO in the Fisher-Rao geometry (NeurIPS 2026 · similarity 0.87)
- Simple Policy Optimization (ICML 2025 · similarity 0.87)
- Colored Noise in PPO: Improved Exploration and Performance through Correlated Action Sampling (AAAI 2024 · similarity 0.86)
- TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing (NeurIPS 2026 · similarity 0.86)
- Absolute Policy Optimization: Enhancing Lower Probability Bound of Performance with High Confidence (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.