Research map / Multi-agent systems and games / Cooperative multi-agent RL
Open Ad Hoc Teamwork with Cooperative Game Theory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalizes open ad hoc teamwork as a dynamic coalitional affinity game and derives CIAO, a graph-based RL method (extending GPL with a star or complete affinity graph, constraints and regularizers) that outperforms GPL on LBF and Wolfpack.
Problem
Open ad hoc teamwork, where a learner must collaborate with a changing number of teammates of unknown types without prior coordination; existing graph-based policy learning (GPL) lacks a principled explanation of its joint Q-value representation.
Approach
Model OAHT as an open stochastic Bayesian coalitional affinity game with a dynamic affinity graph and a dynamic variational strict core solution concept; this justifies GPL's pairwise-plus-individual joint Q-value and yields CIAO, which adds non-negativity constraints on pairwise utilities and a graph-dependent regularizer on individual utilities.
Key results
- CIAO-C outperforms GPL in most Wolfpack and LBF scenarios with max 5 and 9 agents, trained with max 3 agents.
- Ablations removing the non-negative pairwise utility constraint (CIAO-C-NP, CIAO-S-NP) perform notably worse than CIAO-C and CIAO-S.
- The star-graph variant CIAO-S is more generalizable to unseen agent-type sets than the complete-graph variant CIAO-C.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CIAO-C instead of GPL | Open ad hoc teamwork · Wolfpack and LBF; maximum of 5 and 9 agents | performance | Helped |
| Use CIAO-S instead of CIAO-C | CIAO · Wolfpack and LBF; maximum team size varies | performance | Mixed |
| Enforce non-negative pairwise utility instead of allowing negative pairwise utility (ablation) | CIAO-S and CIAO-C · Wolfpack and LBF; maximum of 5 and 9 agents | performance | Helped |
| Allow non-zero individual utility instead of enforcing it to zero (ablation) | CIAO-S and CIAO-C · Wolfpack and LBF; maximum of 5 and 9 agents | performance | Helped |
| Use the GPL training loss without filtering transitions where the team grows (ablation) | CIAO-S and CIAO-C · Wolfpack and LBF; maximum of 5 agents | performance | No clear effect |
| Use a star graph instead of a complete graph | CIAO · Wolfpack and LBF; agent-type sets with intersection or mutual exclusion; maximum team size 5 | generalization | Helped |
| Use CIAO instead of GPL | Open ad hoc teamwork · Wolfpack and LBF; agent-type sets with intersection or mutual exclusion; maximum team size 5 | performance | Helped |
Surprising findings
- The star-graph variant is better with fewer agents while the complete graph is better with more agents.
- The simplified GPL training loss (omitting N_t subset N_t+1 transitions) performs approximately as well as the exact variant.
Limitations
- Assumes fully observable scenarios and fixed teammate policies and types (no adaptation of teammates).
- Individual utility non-negativity is a hypothesis rather than derived from theory.
- Joint Q-value restricted to linear decomposition with pairwise and individual terms.
Experimental setup
Datasets: Level-based Foraging Wolfpack
Benchmarks: Level-based Foraging Wolfpack
Metrics: episode return
Baselines: GPL-Q
Builds on: GPL coalitional affinity game fitted Q-learning DQN LSTM
Closest papers by meaning
- Towards Open Ad Hoc Teamwork Using Graph-based Policy Learning (ICML 2021 · similarity 0.94)
- N-agent Ad Hoc Teamwork (NeurIPS 2024 · similarity 0.90)
- ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork (NeurIPS 2026 · similarity 0.90)
- Ad Hoc Teamwork via Offline Goal-Based Decision Transformers (ICML 2025 · similarity 0.89)
- Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents (AAAI 2024 · similarity 0.87)
- Team-Fictitious Play for Reaching Team-Nash Equilibrium in Multi-team Games (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.