Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LangGround, which aligns MARL agents' communication vectors with text embeddings of GPT-4 embodied-agent messages via a supervised cosine loss added to the RL objective, enabling human-interpretable communication in Predator Prey and USAR.
Problem
Communication learned by multi-agent RL is usually not interpretable to humans or unseen teammates, limiting ad-hoc teamwork; prior interpretable-communication methods are rarely evaluated in interactive team tasks with ad-hoc human-agent teams.
Approach
Ground MARL communication vectors on synthetic (observation, action, message) data from GPT-4 embodied agents by adding a cosine-similarity supervised loss to the IC3Net RL loss, then translate communication vectors to natural language by cosine-similarity lookup in the dataset.
Key results
- In USAR, LangGround converges in fewer steps than all baselines with lower variance, and in Predator Prey it matches IC3Net and aeComm on final performance.
- LangGround reaches topographic similarity 0.67 vs 0.54 for IC3Net, 0.37 for aeComm, and 0.35 for protoComm in pp_v0.
- In ad-hoc teams with one GPT-4 agent, LangGround+LLM beat noComm+LLM and aeComm+LLM in pp_v1 and pp_v0 (e.g., 8.5 vs 10.6 in pp_v1), but not in USAR (23.2 vs 20.3 for aeComm+LLM).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Align communication vectors with LLM-generated natural-language embeddings using supervised grounding | LangGround · Predator Prey pp_v0 | topographic similarity ρ: 0.54±0.14 -> 0.67±0.07; 0.37±0.05 -> 0.67±0.07; 0.35±0.35 -> 0.67±0.07 | Helped |
| Use language grounding during MARL communication learning | LangGround · USAR | episode length to task completion | Helped |
| Use language grounding during MARL communication learning | LangGround · Predator Prey pp_v1 and pp_v0 | episode length to task completion | No clear effect |
| Use language grounding during MARL communication learning | LangGround · Predator Prey 10 by 10 | task performance | Helped |
| Align agent communications with language embeddings | LangGround · Predator Prey pp_v0 | cosine similarity and BLEU | Helped |
| Train with language grounding on a subset of states | LangGround · Predator Prey 10 by 10 | team performance | Helped |
| Use LangGround communication instead of aeComm communication with an unseen LLM teammate | Ad-hoc team · Predator Prey pp_v1 | steps to complete task: 10.3 ± 6.46 -> 8.5 ± 5.76 | Helped |
| Use LangGround communication instead of no communication with an unseen LLM teammate | Ad-hoc team · Predator Prey pp_v0 | steps to complete task: 20.0 ± 0.00 -> 15.5 ± 4.80 | Helped |
| Use LangGround communication instead of no communication with an unseen LLM teammate | Ad-hoc team · USAR | steps to complete task: 32.4 ± 13.47 -> 23.2 ± 10.61 | Helped |
| Remove the communication gating function (ablation) | LangGround · Task environments in the gating ablation | task performance | Hurt |
| Increase supervised-loss weight too high (ablation) | LangGround · Task environments in the loss-weight ablation | task performance | Hurt |
| Increase RL-loss weight too high (ablation) | LangGround · Task environments in the loss-weight ablation | communication alignment | Hurt |
Surprising findings
- Language grounding did not reduce task performance and even accelerated communication emergence, contrary to prior reports that reconstruction losses hurt RL performance.
- In ad-hoc teams, LangGround+LLM performed worse than the homogeneous LLM team and in pp_v1 the ad-hoc result was not better than the LLM-only baseline.
Limitations
- Ad-hoc evaluation uses GPT-4 agents to emulate humans and only 24 episodes per condition over 3 seeds, with large standard deviations.
- Communication alignment relies on a static offline dataset and nearest-embedding lookup; the authors leave online LLM querying to future work.
- Small grid-world environments (5x5 Predator Prey, five-room USAR) with a single LLM team setup limit generality.
Experimental setup
Datasets: LangGround dataset generated by GPT-4 embodied agents in Predator Prey and USAR
Benchmarks: Predator Prey (5x5, vision 0 and 1) Predator Prey 10x10 USAR
Metrics: episode length / steps to task completion cosine similarity BLEU topographic similarity ad-hoc teamwork steps
Baselines: IC3Net aeComm VQ-VIB protoComm noComm
Builds on: IC3Net REINFORCE CICERO
Closest papers by meaning
- Learning to Ground Multi-Agent Communication with Autoencoders (NeurIPS 2021 · similarity 0.91)
- Embodied, Intelligent Communication for Multi-Agent Cooperation (AAAI 2023 · similarity 0.90)
- Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data (NeurIPS 2025 · similarity 0.89)
- LLM-Assisted Semantically Diverse Teammate Generation for Efficient Multi-agent Coordination (ICML 2025 · similarity 0.87)
- Building Cooperative Embodied Agents Modularly with Large Language Models (ICLR 2024 · similarity 0.87)
- Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.