Research map / Multi-agent systems and games / Cooperative multi-agent RL
Multi-Agent Coordination via Multi-Level Communication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeqComm, a multi-level communication scheme where agents make decisions asynchronously in a negotiation-determined priority order, sharing hidden states and then actual upper-level actions, built on MAPPO with a learned world model.
Problem
Coordination in cooperative multi-agent RL under partial observability: synchronous decision-making causes circular dependencies, so agents cannot communicate actual actions and may miscoordinate or get trapped by relative overgeneralization.
Approach
Treat agents asynchronously: a negotiation phase ranks agents by intention values computed from a learned world model and critic, then upper-level agents act first and broadcast actual actions to lower-level agents in a launching phase, while still executing simultaneously.
Key results
- SeqComm outperforms communication-free MAPPO and communication-based TarMAC on nine SMACv2 maps (protoss, terran, zerg at 5v5, 10v10, 10v11) with five seeds.
- Local-communication SeqComm beats the Random-priority and No-action ablations in SMACv2 maps, showing priority ordering matters.
- Theory shows monotonic improvement and convergence of sequential agent-by-agent PPO regardless of decision order.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SeqComm instead of TarMAC | MAPPO on SMACv2 · SMACv2, nine maps | win rate | Helped |
| Add communication instead of using communication-free MAPPO | MAPPO on SMACv2 · SMACv2, nine maps | win rate | Helped |
| Use full communication instead of local communication | SeqComm on SMACv2 · SMACv2, nine maps | win rate | Helped |
| Determine decision priority with SeqComm instead of random priority at each timestep (ablation) | Local SeqComm on SMACv2 · SMACv2, all maps | win rate | Helped |
| Determine decision priority with SeqComm instead of giving agents only others' observations (ablation) | Local SeqComm on SMACv2 · SMACv2, all maps | win rate | Helped |
| Use random decision priority instead of observation-only communication (ablation) | Local SeqComm on SMACv2 · SMACv2, most tasks | win rate | Hurt |
| Increase local communication range (ablation) | Local SeqComm · SMACv2 | performance | Helped |
| Make agent A decide before agent B instead of agent B deciding first or both deciding simultaneously | MAPPO in the one-step matrix game · One-step matrix game | mean reward | Helped |
| Learn the decision priority instead of fixing it as B → A or using simultaneous decisions | MAPPO in the one-step matrix game · One-step matrix game | mean reward | Helped |
Surprising findings
- Random decision-priority ordering often performs worse than no action communication at all.
- Learning the decision order directly is infeasible as it scales exponentially with agent count and does not generalize to varying agent numbers.
Limitations
- Assumes access to local observations of any other agent in the full version, which may be unrealistic.
- Full communication incurs substantial communication overhead.
- Theoretical bound depends on world-model error, which may be large in practice.
Experimental setup
Datasets: SMACv2
Benchmarks: SMACv2
Metrics: win rate
Baselines: MAPPO TarMAC Random priority ablation No action ablation
Builds on: MAPPO TarMAC Stackelberg equilibrium RL TRPO
Closest papers by meaning
- Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation (AAAI 2025 · similarity 0.89)
- Multi-Agent Reinforcement Learning with Communication-Constrained Priors (NeurIPS 2025 · similarity 0.88)
- Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximation (ICML 2023 · similarity 0.88)
- Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning (ICLR 2025 · similarity 0.87)
- CoDe: Communication Delay-Tolerant Multi-Agent Collaboration via Dual Alignment of Intent and Timeliness (AAAI 2025 · similarity 0.87)
- Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.