Research map / Multi-agent systems and games / Human-AI and goal-conditioned

NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming

NeurIPS 2026 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Human-AI coordination agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NestRL, a nested training regime that trains level-2 robot agents against adaptive level-1 partners (themselves trained against fixed level-0 policies) in a multi-recipe Overcooked variant, to support mutual adaptation in human-AI teaming.

Problem

Human-AI teaming requires agents to coordinate with partners who adapt their strategies in response to the agent, while standard MARL and diversity-based training converge to static or brittle conventions that generalize poorly to adaptive partners.

Approach

Formulate mutual adaptation as a bounded level-2 I-POMDP and train agents in a nested fashion: level-1 partners are trained against fixed level-0 policies, then a level-2 robot is trained against these adaptive partners, with RNN latent belief embeddings trained end-to-end with PPO.

Key results

• NestRL achieves 81.3% average success with held-out adaptive L1 partners vs 60.0% for LILI, 55.0% for PACE and 54.7% for Generalist, with lower sensitivity to partner identity (CV 0.108).
• Self-play agents reach 92% self-play success but average at most 35% cross-play with near-zero recipe entropy (H<0.15), showing convention collapse.
• In a user study with N=25, NestRL achieves 79.5% episode delivery rate vs 51.8% for PACE (paired t(21)=2.41, p=.025).

Surprising findings

• Self-play agents trained with IPPO collapse to incompatible conventions, with even same-recipe agents failing to coordinate due to task-irrelevant signalling conventions.

Limitations

• Framework assumes partners adapt via recursive reasoning; irrational or non-recursive human adaptation is not modeled.
• Level-3 and higher partner reasoning is not modeled.
• Assumes shared reward function across agents.

Experimental setup

Datasets: multi-recipe Overcooked (PO-Overcooked) environment rule-based skill-based peer pool

Benchmarks: multi-recipe Overcooked (PO-Overcooked)

Metrics: success rate coefficient of variation recipe entropy cross-play success NASA-TLX delivery rate

Baselines: LIAM LILI Generalist PACE IPPO

Builds on: I-POMDP IPPO LIAM LILI PACE

Closest papers by meaning

• Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.90)
• Beyond Single Stationary Policies: Meta-Task Players as Naturally Superior Collaborators (NeurIPS 2024 · similarity 0.89)
• Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.89)
• LLM-Assisted Semantically Diverse Teammate Generation for Efficient Multi-agent Coordination (ICML 2025 · similarity 0.89)
• Partner Modelling Emerges in Recurrent Agents (But Only When It Matters) (NeurIPS 2025 · similarity 0.88)
• Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems (NeurIPS 2024 · similarity 0.88)