Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Learning to Cooperate with Humans using Generative Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GAMMA, a VAE-based generative model of partner behavior trained on simulated populations or human data, used to sample diverse partners for training a PPO Cooperator in Overcooked, with a human-adaptive sampling variant evaluated in a 80-user study.
Problem
Zero-shot coordination with real humans fails when simulated partner populations or limited human data do not cover the diversity of human strategies in cooperative MARL.
Approach
Learn a VAE latent-variable generative model of partner behavior from any interaction data (simulated populations or human data), sample latent partners to train a PPO Cooperator, and steer sampling toward humans with a human-adaptive latent prior estimated from a small amount of human data.
Key results
- GAMMA HA reaches 91.11 on Counter Circuit and 93.09 on Multi-strategy Counter in the real-human study, versus best baselines of 76.19 (MEP) and 85.26 (PPO+BC).
- MEP+GAMMA scores 88.30 on Multi-strategy Counter vs. 64.02 for MEP, reported as a 38% improvement.
- CoMeDi+GAMMA still fails to learn tomato strategies on Multi-strategy Counter, with scores of 34.72 versus 27.11 for CoMeDi.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the Cooperator with a generative partner model trained on FCP-generated population data | FCP · Counter Circuit | cooperative score: 26.24 $\pm$ 0.96 -> 57.87 $\pm$ 1.35 | Helped |
| Train the Cooperator with a generative partner model trained on FCP-generated population data | FCP · Multi-strategy Counter | cooperative score: 34.35 $\pm$ 1.47 -> 65.36 $\pm$ 1.24 | Helped |
| Train the Cooperator with a generative partner model trained on CoMeDi-generated population data | CoMeDi · Counter Circuit | cooperative score: 56.87 $\pm$ 1.43 -> 72.17 $\pm$ 1.61 | Helped |
| Train the Cooperator with a generative partner model trained on CoMeDi-generated population data | CoMeDi · Multi-strategy Counter | cooperative score: 27.11 $\pm$ 1.46 -> 34.72 $\pm$ 1.65 | Helped |
| Train the Cooperator with a generative partner model trained on MEP-generated population data | MEP · Counter Circuit | cooperative score: 76.19 $\pm$ 1.89 -> 81.10 $\pm$ 2.03 | Helped |
| Train the Cooperator with a generative partner model trained on MEP-generated population data | MEP · Multi-strategy Counter | cooperative score: 64.02 $\pm$ 2.07 -> 88.30 $\pm$ 2.51 | Helped |
| Replace the human-data behavior-cloning partner with a generative model trained on human data | PPO + BC · Counter Circuit | cooperative score: 53.51 $\pm$ 1.37 -> 59.67 $\pm$ 1.35 | Helped |
| Replace the human-data behavior-cloning partner with a generative model trained on human data | PPO + BC · Multi-strategy Counter | cooperative score: 85.26 $\pm$ 2.28 -> 77.53 $\pm$ 2.00 | Hurt |
| Add human-adaptive latent sampling using MEP population data and human data | GAMMA · Counter Circuit | cooperative score: 53.51 $\pm$ 1.37 -> 91.11 $\pm$ 2.96 | Helped |
| Add human-adaptive latent sampling using MEP population data and human data | GAMMA · Multi-strategy Counter | cooperative score: 85.26 $\pm$ 2.28 -> 93.09 $\pm$ 3.19 | Helped |
| Use full fine-tuning instead of decoder-only fine-tuning with human data (ablation) | GAMMA HA · Multi-strategy Counter | cooperative score: 103.76 $\pm$ 1.96 -> 34.05 $\pm$ 2.01 | Hurt |
Surprising findings
- Training on human data (PPO+BC) was the best baseline on the more complex layout, contrary to prior results that simulated populations outperform it.
- GAMMA built on a CoMeDi population performed poorly on Multi-strategy Counter because the source population lacked diverse strategies, illustrating 'garbage in, garbage out'.
Limitations
- Performance depends on the diversity and quality of the source cooperation data; low-quality populations give little benefit.
- Human study participants were recruited from Prolific and may not be representative; the human dataset is small.
- Only two-player settings were studied; scaling to more agents would require better sampling techniques.
Experimental setup
Datasets: Overcooked human-human dataset (Carroll et al.) simulated populations from FCP, MEP, CoMeDi newly collected Multi-strategy Counter human trajectories
Benchmarks: Overcooked (Cramped Room, Asymmetric Advantages, Coordination Ring, Forced Coordination, Counter Circuit, Multi-strategy
Metrics: episode reward / cooperative score Likert subjective ratings Holm-Bonferroni corrected p-values
Baselines: FCP MEP CoMeDi PPO-BC HSP
Builds on: fictitious co-play maximum entropy population-based training CoMeDi PPO HSP
Closest papers by meaning
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.90)
- Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination (ICML 2025 · similarity 0.89)
- Partner Modelling Emerges in Recurrent Agents (But Only When It Matters) (NeurIPS 2025 · similarity 0.88)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.88)
- Melting Pot Contest: Charting the Future of Generalized Cooperative Intelligence (NeurIPS 2024 · similarity 0.88)
- Diversity Is Not All You Need: Training A Robust Cooperative Agent Needs Specialist Partners (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.