Research map / Multi-agent systems and games / Cooperative multi-agent RL
Difference Advantage Estimation for Multi-Agent Policy Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace GAE advantage (beta=0) with DAE using beta around 0.6 to 0.9 (credit assignment via potential-based difference rewards) | MAPPO advantage estimation · SMAC (easy, hard, super hard maps) | test win rate | Helped |
| Set beta=1 (pure difference rewards / aristocrat utility advantage) (ablation) | MAPPO advantage estimation · SMAC | test win rate | Hurt |
| Set beta=1 instead of 0.95 in MPE tasks (ablation) | MAPPO advantage estimation · MPE cooperative navigation, line control, formation control | episode reward | Mixed |
| Increase beta from 0 to 1 on the 16-step matrix game (ablation) | MAPPO advantage estimation · 16-step matrix game | episode reward | Helped |
| Replace GAE with COMA counterfactual advantage (ablation) | MAPPO · SMAC | test win rate | Hurt |
| Increase beta to 0.9 on hard/easy SMAC maps (ablation) | DAE · SMAC easy and hard maps | test win rate | Mixed |
| Tune beta down with task difficulty (beta=0.6 best on hard, 0.3 best on super hard) (ablation) | DAE · SMAC hard and super hard maps | test win rate | Helped |
| Use DAE (beta in 0 to 1) instead of GAE (ablation) | MAPPO · SMAC | standard deviation of advantages | No clear effect |
Closest papers by meaning
- Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning (NeurIPS 2020 · similarity 0.89)
- STAS: Spatial-Temporal Return Decomposition for Solving Sparse Rewards Problems in Multi-agent Reinforcement Learning (AAAI 2024 · similarity 0.86)
- COSAC: Counterfactual Credit Assignment in Sequential Cooperative Teams (NeurIPS 2026 · similarity 0.86)
- Fair Cooperation in Mixed-Motive Games via Conflict-Aware Gradient Adjustment (NeurIPS 2025 · similarity 0.86)
- Multi-agent cooperation through learning-aware policy gradients (ICLR 2025 · similarity 0.85)
- AgentMixer: Multi-Agent Correlated Policy Factorization (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.