Research map / Multi-agent systems and games / Cooperative multi-agent RL
Learning Multi-Agent Communication with Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CACL, a decentralized MARL communication method that applies SupCon-style contrastive learning across agents' sent and received messages within a time window, grounding messages as views of the same state.
Problem
Learning an effective common communication protocol among independent, decentralized agents in partially observable MARL is hard due to large communication space, high RL variance, and lack of common grounding.
Approach
Treat each agent's message as an incomplete view of the same environment state; use a supervised contrastive (SupCon) loss where messages from the same trajectory within a sliding time window are positives and messages from other trajectories are negatives, training only the message head alongside an IAC RL loss.
Key results
- CACL outperforms IAC, DIAL, PL and AEComm in final performance and learning speed on Traffic-Junction, Predator-Prey and Find-Goal.
- Protocol symmetry on Predator-Prey is 0.95 ± 0.01 for CACL versus 0.89 ± 0.01 for AEComm and 0.66 ± 0.07 for DIAL.
- Goal Location probing accuracy in Find-Goal is 91.28% for CACL versus 76.14% for AEComm and 78.31% for PL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CACL instead of AEComm | communication learning in Predator-Prey · Predator-Prey | Two-Preys capture rate: 51.67% ± 7.60 -> 66.67% ± 7.86 | Helped |
| Use CACL instead of AEComm | communication learning in Find-Goal · Find-Goal | Goal Location accuracy: 76.14% ± 3.36 -> 91.28% ± 1.71 | Helped |
| Use CACL instead of AEComm | communication learning in Predator-Prey · Predator-Prey | protocol symmetry: 0.89 ± 0.01 -> 0.95 ± 0.01 | Helped |
| Use CACL instead of AEComm | communication learning in FindGoal · FindGoal | protocol symmetry: 0.85 ± 0.02 -> 0.92 ± 0.01 | Helped |
| Use CACL instead of AEComm | communication learning in Traffic Junction · Traffic Junction | protocol symmetry: 0.80 ± 0.01 -> 0.98 ± 0.002 | Helped |
| Use a one-positive SimCLR-style objective instead of SupCon's multiple positives (ablation) | CACL · Predator-Prey | evaluation reward: 1.36 ± 9.46 | Hurt |
| Remove the temporal sliding window (window size 1) (ablation) | CACL · Predator-Prey | evaluation performance | Hurt |
| Add DIAL reward gradients to the CACL message head (ablation) | CACL · Find-Goal | episode length | No clear effect |
| Add DIAL reward gradients to the CACL message head (ablation) | CACL · Predator-Prey and Traffic-Junction | task performance | Hurt |
| Add DIAL reward gradients to the AEComm message head (ablation) | AEComm · Predator-Prey and Traffic-Junction | task performance | Hurt |
Surprising findings
- Augmenting CACL or AEComm with DIAL (backpropagating RL gradients into the message head) generally hurts performance, suggesting SSL objectives clash with reward-oriented gradients.
- Using a SimCLR-style single positive per message instead of SupCon yields much worse performance (1.36 ± 9.46).
Limitations
- Evaluated only in fully cooperative settings; less cooperative/adversarial settings unexplored.
- SSL objectives are ineffective when combined with reward-oriented gradients (DIAL), not fully understood.
- Zero-shot cross-play performance is poor; zero-shot communication with novel partners remains unsolved.
Experimental setup
Datasets: Traffic-Junction Predator-Prey Find-Goal
Benchmarks: Traffic-Junction Predator-Prey Find-Goal
Metrics: success rate evaluation episodic reward episode length protocol symmetry probing classification accuracy zero-shot cross-play reward
Baselines: IAC DIAL PL AEComm
Builds on: SupCon independent actor critic (IA2C) autoencoded communication (AEComm) DIAL positive listening
Closest papers by meaning
- Learning Efficient and Interpretable Multi-Agent Communication (ICLR 2026 · similarity 0.90)
- Multi-Agent Reinforcement Learning with Communication-Constrained Priors (NeurIPS 2025 · similarity 0.90)
- Learning to Ground Multi-Agent Communication with Autoencoders (NeurIPS 2021 · similarity 0.90)
- Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation (AAAI 2025 · similarity 0.87)
- Learning Multi-Agent Communication from Graph Modeling Perspective (ICLR 2024 · similarity 0.87)
- Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.