Research map / Multi-agent systems and games / Cooperative multi-agent RL
Learning Multi-Agent Communication from Graph Modeling Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CommFormer, which treats the inter-agent communication architecture as a learnable graph optimized end-to-end via bi-level optimization with Gumbel-based k-hot relaxation, within a transformer encoder-decoder MARL policy trained with PPO.
Problem
Multi-agent reinforcement learning under bandwidth constraints needs to decide who communicates with whom; full broadcast is costly and manually pre-defined communication architectures show high variance across designs.
Approach
Relax the discrete communication adjacency matrix into a continuous learnable graph, optimized jointly with the policy/encoder weights via alternating bi-level gradient updates, with a k-hot Gumbel-max sampler enforcing a fixed sparsity budget (each agent communicates with S×N agents) and attention-based credit allocation over received messages.
Key results
- With sparsity S=0.4 (40% of edges), CommFormer reaches 100.0 win rate on many SMAC maps, matching the fully connected variant on most tasks and outperforming MAT, MAPPO, HAPPO and QMIX in the reported tables.
- On GRF academy 3v2, CommFormer achieves 100.0 success rate vs 98.2 for MAGIC, with 25.4 steps taken vs 34.3 for MAGIC.
- On Predator-Prey, CommFormer obtains average cumulative reward -0.121 vs -0.386 for MAGIC and takes 4.99 steps vs 10.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CommFormer with a learned communication graph and sparsity 0.4 instead of MAGIC | CommFormer · GRF academy scenario 3 vs. 2 | Success Rate: 98.2 -> 100.0 | Helped |
| Use CommFormer with a learned communication graph and sparsity 0.4 instead of MAGIC | CommFormer · GRF academy scenario 3 vs. 2 | Steps Taken: 34.3 -> 25.4 | Helped |
| Use CommFormer with a learned communication graph and sparsity 0.4 instead of HetNet | CommFormer · Predator-Prey (PP) | Avg. Cumulative R: -0.232 -> -0.121 | Helped |
| Use CommFormer with a learned communication graph and sparsity 0.4 instead of HetNet | CommFormer · Predator-Prey (PP) | Steps Taken: 8.30 -> 4.99 | Helped |
| Use CommFormer with a learned communication graph and sparsity 0.4 instead of HetNet | CommFormer · Predator-Capture-Prey (PCP) | Avg. Cumulative R: -0.364 -> -0.197 | Helped |
| Use CommFormer with a learned communication graph and sparsity 0.4 instead of HetNet | CommFormer · Predator-Capture-Prey (PCP) | Steps Taken: 9.98 -> 7.61 | Helped |
| Increase communication graph sparsity (ablation) | CommFormer · SMAC tasks | win rate | Helped |
| Reduce communication graph sparsity (ablation) | CommFormer · SMAC tasks | win rate | Hurt |
| Replace the learned communication graph with a manually pre-defined random architecture (ablation) | CommFormer · SMAC tasks | win rate | Mixed |
Surprising findings
- Pre-defined communication architectures show large variance across random seeds, and poorly chosen ones can hurt cooperation relative to no communication.
Limitations
- Sparsity budget S is a fixed hyperparameter; performance degrades at lower S, especially on tasks with many agents.
- Evaluated only in four simulated environments with few agents in several tasks; no real-world or wireless deployment tested.
- Some reported baseline numbers are copied from prior papers rather than re-run under identical settings, per the table structure.
Experimental setup
Datasets: SMAC Predator-Prey Predator-Capture-Prey Google Research Football
Benchmarks: SMAC Predator-Prey Predator-Capture-Prey Google Research Football
Metrics: win rate average cumulative reward steps taken
Baselines: MAPPO HAPPO QMIX UPDeT MAT SMS TarMAC NDQ MAGIC QGNN CommNet I3CNet
Builds on: MAT PPO HAPPO Gumbel-Max trick DARTS
Closest papers by meaning
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.90)
- Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck (AAAI 2024 · similarity 0.90)
- Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning (AAAI 2025 · similarity 0.90)
- Expressive Multi-Agent Communication via Identity-Aware Learning (AAAI 2024 · similarity 0.89)
- G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networks (ICML 2025 · similarity 0.89)
- Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.