Research map / Multi-agent systems and games / Cooperative multi-agent RL
Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ExpoComm, a MARL communication protocol that uses time-varying exponential graphs as the communication topology, with memory-based message processors and auxiliary grounding losses, to scale communication to many agents.
Problem
Communication in cooperative MARL becomes hard to scale to many agents: learning useful pairwise links is difficult and communication overhead grows quadratically with agent count.
Approach
Replace learned pairwise links with a rule-based time-varying exponential graph (small diameter, O(N log N) or O(N) edges), process messages with GRU/attention memory blocks so information spreads across all agents within log2(N) steps, and ground messages with a global-state prediction or contrastive InfoNCE auxiliary loss.
Key results
- On IMP N=100 Uncorrelated, ExpoComm (K=ceil(log2 N)) reaches 27.81% normalized reward versus 18.36% for ER and 12.86% for QMIX.
- ExpoComm with one-peer graph (K=1) achieves the best performance in most of the twelve scenarios while communication cost grows linearly with agent count.
- ExpoComm shows zero-shot transfer from trained agent counts to larger counts in Battle, performing comparably to ER and better than other baselines under small budgets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ExpoComm K=1 instead of QMIX without communication | QMIX · IMP Correlated N=50 | normalized discounted reward (%): 24.81 -> 43.82 | Helped |
| Use ExpoComm K=1 instead of QMIX without communication | QMIX · IMP Uncorrelated N=100 | normalized discounted reward (%): 12.86 -> 27.34 | Helped |
| Use ExpoComm K=1 instead of QMIX without communication | QMIX · IMP OWF N=100 | normalized discounted reward (%): 65.55 -> 65.26 | Mixed |
| Replace ER K=1 communication topology with ExpoComm K=1 | QMIX with communication · IMP Correlated N=100 | normalized discounted reward (%): -65.14 -> 19.17 | Helped |
| Replace ER K=⌈log₂N⌉ communication topology with ExpoComm K=⌈log₂N⌉ | QMIX with communication · IMP Uncorrelated N=100 | normalized discounted reward (%): 18.36 -> 27.81 | Helped |
| Use ExpoComm K=⌈log₂N⌉ instead of CommNet | QMIX communication · IMP OWF N=100 | normalized discounted reward (%): 34.71 -> 66.23 | Helped |
| Remove memory blocks from message generators (ablation) | ExpoComm · MAgent AdversarialPursuit | task performance | Hurt |
| Remove auxiliary loss for message grounding (ablation) | ExpoComm · MAgent and IMP | task performance | Hurt |
| Use exponential communication topology instead of ER random topology | ExpoComm/ER · IMP benchmarks | normalized discounted reward (%) | Mixed |
| Use ExpoComm for zero-shot transfer to larger agent populations | ExpoComm · MAgent Battle | wins | Helped |
Surprising findings
- Random Erdős–Rényi graphs with the same budget transfer nearly as well as exponential graphs, suggesting global message grounding matters more than topology structure in some settings.
Limitations
- Does not incorporate agent heterogeneity or environment MDP structure into topology construction.
- Evaluated mainly in fully cooperative tasks; partially competitive settings remain untested.
- Only exponential graphs explored; finer topologies and temporal sparsity left open.
Experimental setup
Datasets: MAgent AdversarialPursuit MAgent Battle IMP Uncorrelated IMP Correlated IMP OWF
Benchmarks: MAgent AdversarialPursuit MAgent Battle IMP Uncorrelated IMP Correlated IMP OWF
Metrics: mean episode return normalized discounted reward relative to heuristic policies win rate in zero-shot transfer
Baselines: IDQN QMIX DGN+TarMAC Erdős–Rényi graph communication CommFormer CommNet
Builds on: QMIX IDQN CommFormer TarMAC EPyMARL
Closest papers by meaning
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.89)
- Multi-Agent Reinforcement Learning with Communication-Constrained Priors (NeurIPS 2025 · similarity 0.89)
- Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation (AAAI 2025 · similarity 0.88)
- Deep Reinforcement Learning for Communication Networks (AAAI 2024 · similarity 0.88)
- Learning Multi-Agent Communication from Graph Modeling Perspective (ICLR 2024 · similarity 0.87)
- Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.