Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains model-free GRU agents with PPO to collaborate with diverse partners in Overcooked-AI and shows that partner ability representations emerge in hidden states, but only when the agent can influence partner task allocation.
Problem
Whether flexible collaboration requires explicit agent-modelling mechanisms, or whether partner models can emerge spontaneously in model-free recurrent agents trained only with task reward under cooperative pressure.
Approach
Train simple GRU ego agents with only task reward alongside a distribution of partners with varying subtask speeds, granting the ego agent control over partner subtask allocation; probe hidden states to show partner-ability structure emerges, and that it disappears when influence is removed.
Key results
- Multi-partner influential RNN achieved a blind-agent average throughput of 9.43 soups per episode versus 5.8 for the single-partner RNN and 1.08 for the MLP in the blind-agent experiment.
- Linear probes decoding partner task speeds from hidden states reach highest accuracy for the full multi-partner RNN, while the non-influential RNN is significantly impaired, even below the single-partner RNN.
- In online switching experiments, throughput recovers after the partner switch at t=300 across three layouts, and hidden-state embeddings shift to match the post-switch partner distribution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train across partners with diverse competencies instead of a single partner (ablation) | Blind RNN · Overcooked-AI (five layouts, blind agents) | average throughput (soups per episode): 5.8 -> 9.43 | Helped |
| Use a recurrent agent instead of a memoryless MLP (ablation) | Blind multi-partner agent · Overcooked-AI (five layouts, blind agents) | average throughput (soups per episode): 1.08 -> 9.43 | Helped |
| Give the RNN control over partner behaviour through task allocation (ablation) | Multi-partner RNN · Overcooked-AI (five layouts) | linear probe accuracy for partner task abilities | Helped |
| Train with diverse partners instead of only a single partner (ablation) | RNN · Overcooked-AI (five layouts) | linear probe accuracy for partner task abilities | Helped |
| Train with diverse partners instead of only a single partner (ablation) | RNN · Overcooked-AI (five layouts) | episode reward | Helped |
| Use a multi-partner RNN instead of an MLP (ablation) | Overcooked ego agent · FiveByFive | episode reward | Hurt |
Surprising findings
- The non-influential RNN, trained on the full partner distribution, encodes less partner information than the single-partner RNN trained on one fixed partner.
- The MLP achieved the highest reward on the fivebyfive layout, the only exception to the RNN advantage.
Limitations
- Only a single cooperative environment (Overcooked-AI) in main experiments; CoinGame results are preliminary.
- Influence is implemented as strong direct control over partner subtask, unlike nuanced human influence.
- Only simple representations are studied; no hierarchical or recursive modelling of partner perception, and only two-agent teams are tested.
Experimental setup
Datasets: Overcooked-AI layouts (cramped room, fivebyfive, coord ring, cramped room v3, cramped room v4) JaxMARL CoinGame (appendix)
Benchmarks: Overcooked-AI (5 layouts, unseen partners) CoinGame (appendix)
Metrics: episode reward (soups delivered) throughput linear probe accuracy UMAP structure correlation of task allocation with partner speed profile
Baselines: feedforward MLP single-partner RNN non-influential RNN
Builds on: PPO GRU Overcooked-AI JaxMARL CoinGame Machine Theory of Mind
Closest papers by meaning
- Mental-Models for Multi-Agent Systems (NeurIPS 2026 · similarity 0.89)
- Learning “Partner-Aware” Collaborators in Multi-Party Collaboration (NeurIPS 2025 · similarity 0.89)
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.89)
- Beyond Single Stationary Policies: Meta-Task Players as Naturally Superior Collaborators (NeurIPS 2024 · similarity 0.88)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.88)
- Learning to Cooperate with Humans using Generative Agents (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.