Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Beyond Single Stationary Policies: Meta-Task Players as Naturally Superior Collaborators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Collaborative Bayesian Policy Reuse (CBPR), which infers the human's current meta-task online and switches among meta-task-playing PPO agents, evaluated in Overcooked against BCP, FCP and SP.
Problem
Human-AI collaboration with non-stationary human partners who switch initiative and strategy; single stationary collaborator policies (BCP, FCP, SP) struggle to adapt.
Approach
Replace a single stationary collaborator with a library of meta-task-playing policies, one per predefined meta-task, and use Bayesian policy reuse with intra-episode and inter-episode beliefs to select the policy matching the human's current meta-task.
Key results
- Across dynamic policy switching settings, CBPR consistently outperforms BCP, FCP and SP in most cases in Overcooked (Figure 3).
- With 25 real human volunteers, CBPR shows significantly higher reward than baselines in most layout comparisons under one-sided Mann-Whitney U test (Figure 5).
- Removing the 'others' meta-task category in Soup Coordination drops reward for low-skill partners from ~669-690 to 40.0 (Table 1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove the others meta-task category from the 3-task setup (ablation) | CBPR · Soup Coordination, low-skill partner | mean reward: 668.9 (49.0) -> 40.0 (59.1) | Hurt |
| Remove the others meta-task category from the 3-task setup (ablation) | CBPR · Soup Coordination, medium-skill partner | mean reward: 717.1 (148.1) -> 607.3 (278.5) | Hurt |
| Remove the others meta-task category from the 3-task setup (ablation) | CBPR · Soup Coordination, high-skill partner | mean reward: 647.7 (159.3) -> 622.8 (205.8) | Hurt |
| Vary the number of predefined meta-tasks while retaining the others category (ablation) | CBPR · Soup Coordination, high-skill partner | mean reward: 620.3 (193.3); 600.7 (234.0); 647.7 (159.3) | Mixed |
| Vary the number of predefined meta-tasks while retaining the others category (ablation) | CBPR · Soup Coordination, medium-skill partner | mean reward: 757.8 (100.3); 735.8 (98.7); 717.1 (148.1) | Hurt |
| Vary the number of predefined meta-tasks while retaining the others category (ablation) | CBPR · Soup Coordination, low-skill partner | mean reward: 689.8 (43.9); 680.5 (51.6); 668.9 (49.0) | Hurt |
| Set the human behavior queue length to l=20 (ablation) | CBPR · Overcooked layouts | mean reward | Helped |
| Vary inter-episode belief weight ρ (ablation) | CBPR · Cramped Room | mean reward | No clear effect |
| Vary inter-episode belief weight ρ (ablation) | CBPR · Soup Coordination | mean reward | Helped |
Surprising findings
- SP agents sometimes chose not to cooperate, yielding zero reward against certain partners.
- BCP agents occasionally became immobilized and failed to collaborate in non-separated layouts.
Limitations
- Meta-tasks are manually defined by rule-based policies, which is time-consuming and requires domain expertise.
- Performance depends on how well predefined meta-tasks cover human behaviors; uncovered behaviors fall into an 'others' category.
- Evaluated only in Overcooked with four layouts and a small number of real human participants (25).
Experimental setup
Datasets: Overcooked human-human trajectories (Carroll et al. 2019) rule-based meta-task trajectories (Yu et al. 2023)
Benchmarks: Overcooked (Cramped Room, Coordination Ring, Asymmetric Advantages, Soup Coordination)
Metrics: mean episodic reward human-subject episodic reward
Baselines: BCP FCP SP
Builds on: Bayesian Policy Reuse Fictitious Co-Play Behavioral Cloning Play Proximal Policy Optimization
Closest papers by meaning
- Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.90)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.89)
- Partner Modelling Emerges in Recurrent Agents (But Only When It Matters) (NeurIPS 2025 · similarity 0.88)
- Learning “Partner-Aware” Collaborators in Multi-Party Collaboration (NeurIPS 2025 · similarity 0.88)
- Mental-Models for Multi-Agent Systems (NeurIPS 2026 · similarity 0.87)
- MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.