Research map / Multi-agent systems and games / Games and equilibria
Mastering Zero-Shot Interactions in Cooperative and Competitive Simultaneous Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Albatross, an AlphaZero-style method that learns a smooth best response logit equilibrium (SBRLE) via a temperature-conditioned proxy model and response model, enabling zero-shot interaction with agents of varying rationality in simultaneous games.
Problem
Zero-shot interaction in simultaneous-move games requires modeling the bounded rationality of other agents, since standard AlphaZero-style self-play assumes optimal opponents and fails to exploit or cooperate with weak agents.
Approach
Model each other agent's strength as a scalar logit temperature, train a proxy model conditioned on temperature to approximate logit equilibria, then train a response model conditioned on the other agents' temperatures to compute a smooth best response (the SBRLE); the temperature is estimated online via maximum likelihood within an episode.
Key results
- On Overcooked with a behavior cloning human proxy, Albatross yields on average 37.6% higher cooperation reward than PECAN (excluding Forced Coordination).
- In Battlesnake, Albatross consistently outperforms AlphaZero against weaker MCTS baselines, with the largest reward gap against weak agents.
- MLE temperature estimation converges to the true value within roughly 20 to 30 time steps in Tron.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Albatross instead of PECAN | Overcooked cooperation with a human behavior-cloning agent · Overcooked benchmark, average across layouts | cooperation reward: 37.6% | Helped |
| Use Albatross instead of other methods | Overcooked cooperation with scripted agents · Overcooked scripted-agent settings | cooperation reward | Mixed |
| Adapt to the opponent using temperature-conditioned response policies | Albatross · Battlesnake competitive games | reward | Helped |
| Use Albatross instead of AlphaZero | Four-player stochastic Battlesnake tournament · Battlesnake, stochastic mode with four agents | TrueSkill score | Helped |
| Use Albatross instead of AlphaZero | Four-player cooperative Tron · Cooperative Tron with weak partners | discounted reward | Helped |
| Increase the proxy-agent temperature | Albatross proxy and response policies · Overcooked Counter Circuit | mutual information | Helped |
| Replace the standard MCTS with fixed-depth search and Logit-equilibrium backup (ablation) | AlphaZero adapted to simultaneous games · Simultaneous games | performance | Helped |
| Use the Nagurney and Zhang step-size schedule instead of the alternatives (ablation) | Stochastic fictitious play · Randomly generated normal-form games | policy and value error | Helped |
| Overestimate the other agent's rationality (ablation) | Albatross temperature input · Overcooked Counter Circuit | reward | Hurt |
| Use an exact temperature input rather than online maximum-likelihood estimation (ablation) | Albatross* · Overcooked layouts with high-temperature proxy partners | reward | Helped |
| Set the proxy temperature to 0 (ablation) | Albatross proxy model · Overcooked self-play | reward | Hurt |
| Use temperature-conditioned response behavior rather than assuming optimal opponents | Albatross compared with AlphaZero · Battlesnake Tron and stochastic two-player mode | reward | Mixed |
Surprising findings
- AlphaZero converges to near-maximum temperature (9.99) in the four-player tournament, so it plays as if all opponents were optimal and cannot exploit weak baseline agents.
Limitations
- Requires observations within an episode to estimate rationality; may fail in very short games.
- Planning in joint action space scales exponentially with number of agents and actions, limiting applicability to large domains.
- Not applicable to environments without a known model (e.g., MeltingPot) since planning is needed.
Experimental setup
Datasets: Overcooked human play dataset (for behavior cloning agent) Battlesnake self-play episodes
Benchmarks: Overcooked Battlesnake (Tron, stochastic 2-player and 4-player modes) cooperative Tron variant
Metrics: cooperation reward discounted reward TrueSkill score temperature estimate
Baselines: AlphaZero PECAN FCP MEP TrajDiv HSP PPO MCTS baseline agents
Builds on: AlphaZero Quantal Response Equilibrium Stochastic Fictitious Play Quantal Stackelberg Equilibrium Fictitious Co-Play
Closest papers by meaning
- Towards convergence to Nash equilibria in two-team zero-sum games (ICLR 2023 · similarity 0.88)
- Neural Auto-Curricula in Two-Player Zero-Sum Games (NeurIPS 2021 · similarity 0.88)
- Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solvers (ICML 2021 · similarity 0.87)
- Aligning Individual and Collective Objectives in Multi-Agent Cooperation (NeurIPS 2024 · similarity 0.87)
- Learning Equilibria in Adversarial Team Markov Games: A Nonconvex-Hidden-Concave Min-Max Optimization Problem (NeurIPS 2024 · similarity 0.87)
- Securing Equal Share: A Principled Approach for Learning Multiplayer Symmetric Games (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.