Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a single self-play Overcooked/Dual Destination cooperator across a procedurally generated distribution of environments (Cross-Environment Cooperation, CEC) and shows it zero-shot coordinates with new partners and new layouts, including real humans.
Problem
Zero-shot coordination with novel partners on novel tasks: partner-diversity methods like PBT/FCP and self-play overfit to a single environment and fail to generalize to new layouts.
Approach
Replace partner diversity with environment diversity: train a single policy via self-play (IPPO) on a broad distribution of procedurally generated tasks, without populations or auxiliary losses, so it learns general cooperative norms that transfer to new partners and layouts.
Key results
- On 100 held-out procedurally generated Overcooked layouts, FCP and E3T receive 0 reward while CEC achieves high cross-play reward (CEC 0.966 normalized on procedural single-task Dual Destination per appendix; CEC reported 0.931 on fixed Dual Destination vs oracle 0.955).
- In a human study with 80 Prolific participants, CEC-Finetune significantly outperforms FCP (p<0.01) and E3T on subjective ratings; CEC has the lowest collision rate, though E3T has higher success rate.
- On Dual Destination partially observable setting, CEC reaches 0.74 cross-play versus 0.61 for population-based methods and 0.03 for naive self-play.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with environment diversity using self-play | CEC on Dual Destination · Dual Destination, novel partners and novel environments | normalized reward: CEC scored 0.931 normalized reward; oracle 0.955 | Helped |
| train with environment diversity using self-play | CEC on Dual Destination · Dual Destination, partially observable, novel partners and novel environments | ZSC performance: 0.74 vs. 0.61 and 0.03 | Helped |
| train with environment diversity using self-play | CEC on Multi-task Dual Destination · novel partners and novel environments | performance: CEC (0.446); PBT (0.005); naive self-play (0.004) | Helped |
| fine-tune CEC on one original layout (ablation) | CEC on Overcooked · five original layouts | cross-play performance | Helped |
| fine-tune CEC on one original layout (ablation) | CEC on Overcooked · 100 held-out procedural layouts | cross-play performance: CEC-FT=41.73 | Hurt |
| add E3T partner-policy randomness to CEC (ablation) | CEC on Overcooked · five original layouts | cross-play performance: CEC=130.51, CEC-E3T=28.21 | Hurt |
| add E3T partner-policy randomness to CEC (ablation) | CEC on Overcooked · 100 held-out procedural layouts | cross-play performance: CEC-FT=41.73, CEC-E3T=58.13 | Helped |
| remove the LSTM recurrent core (ablation) | CEC on Dual Destination · Dual Destination | reward | Hurt |
| use a multi-task version with two valid goal solutions (ablation) | CEC on Dual Destination · novel partners, task seen during training | performance: CEC (0.404) vs. PBT (0.251) and naive self-play (0.083) | Mixed |
Surprising findings
- Training in self-play on environment diversity alone yields agents that coordinate with novel partners, contrary to the prevailing view that self-play is insufficient for ZSC.
- CEC gets lower task reward than E3T with humans but is rated more adaptive, less frustrating, and collides less.
Limitations
- CEC has not converged after 3 billion training steps on most layouts, so the performance ceiling is unknown.
- Human study participants were filtered to English speakers, which may bias results.
- CEC adapts more to partners, which can lower task reward when humans play suboptimal strategies; CEC-Finetune trades off generalization to novel layouts.
Experimental setup
Datasets: Overcooked (JaxMARL) Procedurally generated Overcooked layouts Dual Destination gridworld (procedural)
Benchmarks: Overcooked (5 original layouts) 100 procedurally generated held-out Overcooked layouts Dual Destination human-AI Overcooked study (Counter Circuit, Coordination Ring)
Metrics: cross-play reward success rate human subjective Likert ratings collision count replicator dynamic / empirical game-theoretic analysis
Baselines: IPPO FCP E3T CEC-Finetune PBT
Builds on: IPPO Fictitious Co-Play E3T JaxMARL PPO
Closest papers by meaning
- Diversity Is Not All You Need: Training A Robust Cooperative Agent Needs Specialist Partners (NeurIPS 2024 · similarity 0.90)
- ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination (NeurIPS 2024 · similarity 0.89)
- Learning to Cooperate with Humans using Generative Agents (NeurIPS 2024 · similarity 0.89)
- OvercookedV2: Rethinking Overcooked for Zero-Shot Coordination (ICLR 2025 · similarity 0.89)
- Training Generalizable Collaborative Agents via Strategic Risk Aversion (NeurIPS 2026 · similarity 0.89)
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.