Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Diversity Is Not All You Need: Training A Robust Cooperative Agent Needs Specialist Partners
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes information-theoretic measures of partner diversity, specialization and overfitness, and SpecTRL / SpecTRL DAgger to distill XP-min partners into populations that remove handshaking while keeping diversity and specialization, yielding more robust generalists in multi-recipe Overcooked.
Problem
Training a robust ad-hoc cooperative generalist requires diverse training partners, but XP-min partners overfit via handshakes, and MP-reg/MI regularization reduce specialization.
Approach
Defines diversity as entropy of a characteristic (recipe completion) distribution over a population and specialization as negative average per-policy entropy, then distills XP-min partners into new partners via RL reward maximization plus a DAgger imitation loss to strip sabotage/handshake behavior while preserving diversity and specialization.
Key results
- XP-min + MI + SpecTRL DAgger population yields generalist robustness 0.64 ± 0.01, versus 0.56 ± 0.07 for XP-min.
- SpecTRL reduces overfitness of overfit oracle specialists from 0.35 to 0.16 while robustness rises from 0.74 to 0.78.
- SpecTRL DAgger reduces XP-min overfitness from 0.51 to 0.29 while keeping diversity (1.31 to 1.32) and specialization (-0.13 to -0.14).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SpecTRL distillation to the overfit oracle-specialist population | Overfit oracle specialists · Multi-recipe Overcooked | robustness: 0.74 ± 0.02 -> 0.78 ± 0.01 | Helped |
| Add SpecTRL distillation to XP-min partners | XP-min · Multi-recipe Overcooked | overfitness: 0.51 ± 0.05 -> 0.25 ± 0.02 | Helped |
| Add SpecTRL DAgger distillation to XP-min partners | XP-min · Multi-recipe Overcooked | overfitness: 0.51 ± 0.05 -> 0.29 ± 0.06 | Helped |
| Add SpecTRL DAgger distillation to XP-min + MI | XP-min + MI · Multi-recipe Overcooked | robustness: +0.03 | Helped |
| Add MP-reg during XP-min training | XP-min · Multi-recipe Overcooked | robustness: −0.12 | Hurt |
| Add MI during XP-min training | XP-min · Multi-recipe Overcooked | robustness: ≈ | No clear effect |
| Add both MI and MP-reg during XP-min training | XP-min · Multi-recipe Overcooked | robustness: ≈ | No clear effect |
| Add SpecTRL to XP-min + MI partners | XP-min + MI · Multi-recipe Overcooked | diversity: ⇓0.19 | Hurt |
| Add SpecTRL DAgger to XP-min + MI partners | XP-min + MI · Multi-recipe Overcooked | robustness: ⇑0.03 | Helped |
| Add MI during XP-min training | XP-min · Multi-recipe Overcooked | specialization: ⇓0.50 | Hurt |
| Add MP-reg during XP-min training | XP-min · Multi-recipe Overcooked | diversity: ⇑0.17 | Helped |
| Add MI during XP-min training | XP-min · Multi-recipe Overcooked | diversity: ⇑0.32 | Helped |
Surprising findings
- MP-reg and MI regularization increase partner diversity but reduce specialization, so their generalists are not more robust.
- Using both MP-reg and MI together does not yield a more diverse population than either alone.
Limitations
- Measures depend on domain-knowledge characteristic function f and reward shaping for oracle specialists.
- Evaluated only in a single domain, multi-recipe Overcooked.
- SpecTRL methods depend on quality of source partner populations; repeated distillation does not further reduce overfitness.
Experimental setup
Datasets: multi-recipe Overcooked (simulated)
Benchmarks: multi-recipe Overcooked
Metrics: diversity (entropy of recipe completion) specialization (negative conditional entropy) overfitness robustness (harmonic mean of success rate)
Baselines: self-play XP-min XP-min + MP-reg (CoMeDi) XP-min + MI (LIPO) FCP
Builds on: cross-play minimization (XP-min) LIPO CoMeDi DAgger fictitious co-play
Closest papers by meaning
- Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination (ICML 2025 · similarity 0.90)
- Training Generalizable Collaborative Agents via Strategic Risk Aversion (NeurIPS 2026 · similarity 0.89)
- Learning to Cooperate with Humans using Generative Agents (NeurIPS 2024 · similarity 0.88)
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.87)
- Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents (AAAI 2024 · similarity 0.87)
- Controlling Behavioral Diversity in Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.