Research map / Multi-agent systems and games / Cooperative multi-agent RL
Multi-Agent Reinforcement Learning with Communication-Constrained Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CC-MADDPG, a communication-constrained MARL framework that uses link-status priors and a dual mutual information estimator (JSD lower bound for lossless messages, CLUB upper bound for lossy ones) to shape the global reward, evaluated on MPE tasks under Markov- and distance-based message loss.
Problem
Cooperative multi-agent reinforcement learning under lossy communication, where messages are dropped or corrupted in dynamic, unknown environments, degrading policy learning and robustness.
Approach
Model communication-link status as a binary prior to separate lossless from lossy messages, then use a dual MI estimator (JSD lower bound for lossless, CLUB upper bound for lossy) to add MI-based terms to the global reward, integrated with MADDPG.
Key results
- On Simple_Tag under heavy distance-based communication constraint, CC-MADDPG achieves 138.0 reward versus 1.5 for FC-MADDPG and 68.7 for Dropout-MADDPG (dropout-0.2).
- Ablation on Simple_Tag heavy DBC: full model 138.0 vs 68.7 baseline, 81.7 with maximization-only, 120.4 with minimization-only.
- With 9 agents under heavy DBC on Simple_Tag, CC-MADDPG reaches 83.2 versus 49.0 for dropout-0.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add communication-constrained priors and dual MI reward shaping to MADDPG | MADDPG · Simple_Tag, Heavy DBC (1) | average episode cumulative reward: 1.5$\pm$5.2 -> 138.0$\pm$88.1 | Helped |
| Use a test-environment-matched prior instead of generic dropout-0.2 | CC-MADDPG · Simple_Spread, Medium DBC | average episode cumulative reward: -127.6 -> -98.0 | Helped |
| Use a test-environment-matched prior instead of generic dropout-0.2 | CC-MADDPG · Simple_Reference, Medium DBC | average episode cumulative reward: 73.8 -> 107.5 | Helped |
| Add lossless-message MI maximization (ablation) | CC-MADDPG with dropout-0.2 prior · Simple_Tag, Heavy DBC | average episode cumulative reward: 68.7 -> 81.7 | Helped |
| Add lossy-message MI minimization (ablation) | CC-MADDPG with dropout-0.2 prior · Simple_Tag, Heavy DBC | average episode cumulative reward: 68.7 -> 120.4 | Helped |
| Add both lossless-message MI maximization and lossy-message MI minimization (ablation) | CC-MADDPG with dropout-0.2 prior · Simple_Tag, Heavy DBC | average episode cumulative reward: 68.7 -> 138.0 | Helped |
| Add lossless-message MI maximization (ablation) | CC-MADDPG with dropout-0.2 prior · Simple_Spread, Medium DBC | average episode cumulative reward: -143.4 -> -139.7 | Helped |
| Add lossy-message MI minimization (ablation) | CC-MADDPG with dropout-0.2 prior · Simple_Spread, Medium DBC | average episode cumulative reward: -143.4 -> -141.7 | Helped |
| Add both lossless-message MI maximization and lossy-message MI minimization (ablation) | CC-MADDPG with dropout-0.2 prior · Simple_Spread, Medium DBC | average episode cumulative reward: -143.4 -> -127.6 | Helped |
Surprising findings
- Performance under Markov-based communication constraints varies little across light, medium, and heavy loss levels, which the authors attribute to the MPE episode length and policy robustness.
- CC-MADDPG with the dropout-0.2 prior in some constrained settings exceeds its own unrestricted-communication performance (e.g., 138 vs 134.7 on Simple_Tag).
Limitations
- Only evaluated with MADDPG; value-based extension left to future work.
- Evaluated only on small MPE scenarios with fixed 25-step episodes and 3 agents in the main results.
- Communication-constraint models (Markov matrices, distance thresholds) are simulated and may not capture real-world channels.
Experimental setup
Datasets: Simple_Tag (MPE) Simple_Spread (MPE) Simple_Reference (MPE) Simple_Adversary (MPE)
Benchmarks: Simple_Tag Simple_Spread Simple_Reference Simple_Adversary
Metrics: average episode cumulative reward standard deviation
Baselines: MAIC FC-MADDPG Dropout-MADDPG MADDPG
Builds on: MADDPG CLUB Message-Dropout MAIC Dec-POMDP
Closest papers by meaning
- Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation (AAAI 2025 · similarity 0.90)
- Learning Multi-Agent Communication with Contrastive Learning (ICLR 2024 · similarity 0.90)
- RGMComm: Return Gap Minimization via Discrete Communications in Multi-Agent Reinforcement Learning (AAAI 2024 · similarity 0.90)
- Robust Communicative Multi-Agent Reinforcement Learning with Active Defense (AAAI 2024 · similarity 0.89)
- Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning (ICLR 2025 · similarity 0.89)
- Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.