Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Melting Pot Contest: Charting the Future of Generalized Cooperative Intelligence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Summarizes the Melting Pot Contest 2023, a zero-shot social generalization benchmark for cooperative MARL in mixed-motive games, along with its design, results, team approaches, and lessons learned.
Problem
Evaluating whether learning agents can cooperate with novel co-players in mixed-motive multi-agent settings, i.e., measuring cooperative intelligence beyond environmental generalization.
Approach
Uses the Melting Pot suite with held-out background populations to test zero-shot social generalization of focal populations in resident and visitor modes, scoring per-capita focal return normalized against baselines, and analyzing submissions with Bayesian capability profiles and background-population returns.
Key results
- Over 600 participants across 100+ teams; 73 teams submitted in development phase and 23 teams provided final selections.
- 8 of 23 teams outperformed the best baseline (normalized score above 1.0) on the full 51 held-out scenarios.
- Top-performing focal populations often scored poorly on background-population return, indicating overfitting to in-group focal returns, especially for hard-coded policies.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use substrate-specific hard-coded policies | Contest submissions · Melting Pot contest evaluation | aggregate team ranking | Helped |
| Optimize focal returns with hard-coded or rule-based policies | Contest submissions · Melting Pot held-out scenarios | background population per-capita return | Hurt |
| Use hard-coded or rule-based policies rather than RL-based solutions | Contest submissions · Melting Pot held-out scenarios | overfitting to the focal objective | Hurt |
| Downsample the sprite observations using the provided wrapper (ablation) | Melting Pot contest agents · Melting Pot substrates | performance | No clear effect |
| Use the XGBoost assessor instead of aggregate mean prediction | Capability prediction models · Capability prediction test set | MSE: 0.042 -> 0.012 | Helped |
| Use the measurement layout instead of aggregate mean prediction | Capability prediction models · Capability prediction test set | MSE: 0.042 -> 0.025 | Helped |
Surprising findings
- Hard-coded rule-based policies overfit to the focal objective more than RL solutions and achieved the highest aggregate team ranking.
- High-ranking teams often excelled on a few substrates while severely underperforming on others.
Limitations
- Top solutions overfit to a few substrates and to the single mean focal-return metric.
- Hard-coded solutions exploit differences between focal and background agents, and evaluation did not penalize anti-social outcomes.
Experimental setup
Datasets: Melting Pot substrates (Prisoner's Dilemma in the Matrix: Arena, Clean Up, Allelopathic Harvest, Territory: Rooms) 22 development scenarios 51 held-out scenarios
Benchmarks: Melting Pot Contest 2023 held-out scenarios (51 scenarios across 4 substrates)
Metrics: normalized per-capita focal return background per-capita return inter-quartile mean (IQM) mean squared error of capability predictions
Baselines: Independent PPO (RLlib) Actor-Critic (ACB) V-MPO OPRE Hard-coded policies (Marlshmallows) Cooperative MARL VDN (rookie)
Builds on: Melting Pot PPO VDN IMPALA Legg and Hutter intelligence definition
Closest papers by meaning
- Learning to Cooperate with Humans using Generative Agents (NeurIPS 2024 · similarity 0.88)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.87)
- Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination (ICML 2025 · similarity 0.87)
- Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents (NeurIPS 2024 · similarity 0.86)
- Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.86)
- Aligning Individual and Collective Objectives in Multi-Agent Cooperation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.