Research map / Multi-agent systems and games / Cooperative multi-agent RL
GTDE: Grouped Training with Decentralized Execution for Multi-agent Actor-Critic
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GTDE, a grouped training with decentralized execution paradigm that uses an adaptive Gumbel-Sigmoid grouping module and group information aggregation so actor-critic agents train on local group information instead of joint or individual information.
Problem
Existing DTDE and CTDE multi-agent actor-critic paradigms degrade as agent count grows: DTDE suffers non-stationarity and weak cooperation, while CTDE's joint information hits the curse of dimensionality and redundancy in large-scale partially observable systems.
Approach
Each agent learns from its observation history a distribution over links to other agents, sampled end-to-end with Gumbel-Sigmoid to form dynamic groups; a group information aggregation module (matrix multiplication or GAT) merges member information for the critic, with no centralized module needed.
Key results
- In Gather with 495 agents, GTDE increased total reward by an average of 382% over the baseline (abstract).
- In Battle (64 v 64), GTDE achieved a 100% win rate against IAC and MAAC.
- On SMACv2 20v20, GTDE improved win rate by 6.0% over CTDE and 17.0% over DTDE, while using about 4.9 agent-information nodes on average versus 20 for CTDE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CTDE or DTDE with GTDE | multi-agent actor-critic · SMACv2 20v20 | win rate: 6.0% and 17.0% | Helped |
| replace joint or individual observations with grouped local information | Battle 64v.s.64 · Battle | win rate: 100% | Helped |
| use adaptive rather than uniform grouping (ablation) | GTDE · Gather | total reward: 11.6% | Helped |
| use adaptive rather than fixed grouping (ablation) | GTDE · Gather | total reward: 10.1% | Helped |
| use adaptive grouping rather than pairwise links (ablation) | GTDE · Gather | total reward: 18.2% | Helped |
| replace adaptive grouping with fixed grouping (ablation) | GTDE · SMACv2 | win rate: 53.4 -> 49.3; 48.4 -> 42.2; 38.8 -> 32.5 | Hurt |
| replace adaptive grouping with uniform grouping (ablation) | GTDE · SMACv2 | win rate: 53.4 -> 46.4; 48.4 -> 43.5; 38.8 -> 32.8 | Hurt |
| use fixed grouping rather than pairwise links (ablation) | GTDE · Gather | total reward | No clear effect |
Surprising findings
- GTDE-F (fixed grouping) and GTDE-A (pairwise joint links) achieve similar total reward in Gather, suggesting information redundancy in joint observations.
- In Battle, CTDE-style MAAC performs worse than IAC because its joint input dimension is very large under the same network size.
Limitations
- Scalability issue: adjacency matrix is non-scalable, so new agents can only link to existing agents during training and not to each other.
- Evaluated only in simulators (SMACv2, Battle, Gather) rather than real-world applications.
Experimental setup
Datasets: SMACv2 Battle Gather
Benchmarks: SMACv2 20v20 Battle 64v64 Gather 495 agents
Metrics: win rate total reward average information required per agent
Baselines: MAPPO IPPO MAAC IAC GTDE-F GTDE-U GTDE-A
Builds on: MAPPO IPPO MAAC PPO Gumbel-Softmax graph attention network
Closest papers by meaning
- Intrinsic Action Tendency Consistency for Cooperative Multi-Agent Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Multi-Agent Guided Policy Optimization (ICLR 2026 · similarity 0.87)
- Solving Homogeneous and Heterogeneous Cooperative Tasks with Greedy Sequential Execution (ICLR 2024 · similarity 0.87)
- More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization (ICLR 2023 · similarity 0.87)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.86)
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.