Research map / Multi-agent systems and games / Human-AI and goal-conditioned
OvercookedV2: Rethinking Overcooked for Zero-Shot Coordination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that zero-shot coordination failures in Overcooked stem mainly from poor state coverage, and introduces OvercookedV2 with partial observability, asymmetric information, and stochasticity to create genuine ZSC challenges that existing methods fail to solve.
Problem
Overcooked is widely used to benchmark zero-shot coordination, but the SP-XP gap there may reflect insufficient state coverage rather than true coordination challenges, and the benchmark lacks partial observability, asymmetric information and stochasticity needed for meaningful ZSC evaluation.
Approach
Augment training start states with states collected from cross-play rollouts of independently trained self-play agents (state-augmented self-play) to close the SP-XP gap, then build a new Overcooked variant with view radius, recipe indicators, grounded/arbitrary communication channels and randomized starts that requires test-time protocol formation.
Key results
- State-augmented self-play reduces the Counter Circuit SP-XP gap from 104 to 44 and Coordination Ring from 118 to 71, and nearly closes the Forced Coordination gap (61 to 8).
- On OvercookedV2 coordination layouts, no method closes the SP-XP gap; e.g., Grounded Coordination Simple self-play XP is -55 ± 124 against SP 154 ± 10.
- Other-play improves XP in most OvercookedV2 layouts, e.g., Test Time Wide XP rises from -30 (self-play) to -11, but still well below SP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with augmented starting states (ablation) | PPO on original Overcooked · Coordination Ring | XP score: 96 \pm 72 -> 198 \pm 50 | Helped |
| train with augmented starting states (ablation) | PPO on original Overcooked · Forced Coord. | XP score: 138 \pm 70 -> 193 \pm 21 | Helped |
| train with augmented starting states (ablation) | PPO on original Overcooked · Counter Circuit | XP score: 59 \pm 57 -> 140 \pm 38 | Helped |
| train with augmented starting states (ablation) | PPO on original Overcooked · Cramped Room | XP score: 257 \pm 2 -> 255 \pm 6 | No clear effect |
| train with augmented starting states (ablation) | PPO on original Overcooked · Asymm. Advantages | XP score: 278 \pm 1 -> 276 \pm 1 | No clear effect |
| train with augmented starting states (ablation) | PPO with OvercookedV2 architecture · Grounded Coord. Simple | XP score: -55 \pm 124 -> -17 \pm 115 | Helped |
| train with augmented starting states (ablation) | PPO with OvercookedV2 architecture · Demo Cook Simple | XP score: 36 \pm 82 -> 112 \pm 50 | Helped |
| train with augmented starting states (ablation) | PPO with OvercookedV2 architecture · Test Time Simple | XP score: -81 \pm 99 -> -55 \pm 103 | Helped |
| use ingredient-permutation symmetry (Other-Play) | PPO with OvercookedV2 architecture · Grounded Coord. Simple | XP score: -55 \pm 124 -> 4 \pm 73 | Helped |
| use ingredient-permutation symmetry (Other-Play) | PPO with OvercookedV2 architecture · Test Time Simple | XP score: -81 \pm 99 -> -3 \pm 51 | Helped |
| train a best response to a self-play population (Fictitious Co-Play) | PPO with OvercookedV2 architecture · Test Time Wide | XP score: -30 \pm 96 -> 23 \pm 40 | Helped |
| train a best response to a self-play population (Fictitious Co-Play) | PPO with OvercookedV2 architecture · Demo Cook Wide | XP score: 40 \pm 61 -> 13 \pm 28 | Hurt |
Surprising findings
- Standard self-play agents in Cramped Room and Asymmetric Advantages achieve near-zero SP-XP gap, indicating Overcooked lacks a true coordination challenge in these layouts.
- Fictitious co-play agents score much lower than self-play agents even in self-play evaluation, possibly due to out-of-distribution SP evaluation.
Limitations
- Other-play requires known symmetries beforehand, which must be defined per layout.
- The benchmark cannot represent all ZSC challenges, so evaluation should not be limited to it.
- Fictitious co-play uses a small population size (8 self-play agents), which may limit its performance.
Experimental setup
Datasets: Overcooked layouts (JaxMARL) OvercookedV2 layouts
Benchmarks: Overcooked OvercookedV2 (Grounded Coordination, Test-Time Protocol Formation, Demo Cook layouts)
Metrics: self-play score cross-play score SP-XP gap
Baselines: Self-Play State-Augmented Self-Play Other-Play Fictitious Co-Play
Builds on: PPO Other-Play Fictitious Co-Play JaxMARL Overcooked
Closest papers by meaning
- ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordination (NeurIPS 2024 · similarity 0.89)
- Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination (ICML 2025 · similarity 0.89)
- On the Utility of Learning about Humans for Human-AI Coordination (NeurIPS 2019 · similarity 0.88)
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.86)
- Beyond Single Stationary Policies: Meta-Task Players as Naturally Superior Collaborators (NeurIPS 2024 · similarity 0.86)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.