Research map / Multi-agent systems and games / Cooperative multi-agent RL
Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TGCNet, a value-based MARL method that models inter-agent communication as a learned dynamic directed graph and uses graph coarsening networks to approximate the global state, so training needs no global state input.
Problem
Cooperative MARL in partially observed tasks relies on global states during centralized training and lacks dynamic directed communication mechanisms that minimize communication cost while reducing reliance on global state.
Approach
Represent the communication policy as an adjacency trajectory matrix learned by a multi-key gated (gumbel-softmax) communication network, feed it into a graph coarsening network (GCN + self-attention pooling + readout) to approximate the global state in the mixing network during training, and use a Transformer decoder in the per-agent Q-network during execution.
Key results
- TGCNet's average test win rate in hard SMAC scenarios is reported as nearly 20% higher than the compared algorithms.
- On Hallway, TGCNet exceeds GraphMix by approximately 20%, while QMIX fails.
- On LBF, TGCNet matches GraphMix performance; ablations show TGCNet performs similarly to TGCNet_QMIX, supporting the graph coarsening network as a global-state approximation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TGCNet instead of GraphMix | Hallway · Hallway | task performance: approximately 20% | Helped |
| Use TGCNet instead of GraphMix | LBF · LBF | task performance | No clear effect |
| Use TGCNet instead of other algorithms | SMAC hard scenarios · SMAC hard scenarios | average test win rate: nearly 20% | Helped |
| Replace the graph coarsening network with a QMIX hybrid network (ablation) | TGCNet · one hard and two super hard SMAC scenarios | performance | No clear effect |
| Replace multi-key communication with full communication (ablation) | TGCNet · SMAC corridor scenario | convergence speed | Hurt |
| Replace multi-key communication with full communication (ablation) | TGCNet · SMAC corridor scenario | final performance | No clear effect |
Surprising findings
- QMIX without communication fails on Hallway but performs effectively on LBF because agents can observe nearby grids.
Limitations
- Evaluated only on Hallway, LBF, and SMAC; authors note need for evaluation on more complex real-world domains.
- Communication visualization interpretability is qualitative, based on a single test scenario.
Experimental setup
Datasets: Hallway Level-Based Foraging (LBF) StarCraft Multi-Agent Challenge (SMAC) maps including 1o2r_vs_4r, 1o10b_vs_1r, 27m_vs_30m
Benchmarks: Hallway Level-Based Foraging (LBF) SMAC hard and super hard maps
Metrics: test win rate team reward
Baselines: QMIX IC3Net DGN G2ANet GraphMix
Builds on: QMIX gumbel-softmax EPyMARL self-attention pooling Transformer
Closest papers by meaning
- Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning (AAAI 2025 · similarity 0.91)
- Learning Multi-Agent Communication from Graph Modeling Perspective (ICLR 2024 · similarity 0.90)
- Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck (AAAI 2024 · similarity 0.89)
- Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.