Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Select to Perfect: Imitating desired behavior from large multi-agent data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Exchange Values, a Shapley-like per-agent contribution measure for collective desirability scores, and EV2BC, a behavior cloning variant that imitates only agents with high Exchange Values.
Problem
Imitation learning from large multi-agent datasets that contain undesirable behavior; desirability is scored only per collective trajectory, so individual agents' contributions must be estimated under group-size constraints, incomplete data, and fully anonymized IDs.
Approach
Define Exchange Value as the expected change in group desirability when substituting an agent with a random one (using only same-size groups, unlike Shapley values), estimate it from incomplete or anonymized data via EV-Clustering and behavior embeddings, and imitate only agents whose EV exceeds a threshold.
Key results
- In Overcooked Cramped Room, EV2BC reaches 91.6 vs 54.2 for Group-BC and 10.8 for BC on D^lambda (DVF = soups).
- In Tragedy of the Commons with 120 agents, EV2BC achieves v_final 10576.8 vs 5324.2 for Group-BC and 2693.6 for BC.
- EV-Clustering reduces within-cluster variance in the unobserved trait lambda by about 89% when 50% of data is observed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on Overcooked demonstrations · Cramped Room, D^λ | soups: 54.2 ± 5.45 -> 91.6 ± 12.07 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on Overcooked demonstrations · Cramped Room, D^adv | soups: 64.8 ± 7.62 -> 104.2 ± 10.28 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on Overcooked demonstrations · Cramped Room, D^human | soups: 163.34 ± 6.08 -> 170.89 ± 6.8 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on Overcooked demonstrations · Coordination Ring, D^λ | soups: 24 ± 4.69 -> 30.2 ± 6.91 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on Overcooked demonstrations · Coordination Ring, D^adv | soups: 14.6 ± 2.48 -> 12.4 ± 2.65 | Hurt |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on Overcooked demonstrations · Coordination Ring, D^human | soups: 102.2 ± 6.19 -> 114.89 ± 5.08 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on ToC demonstrations · ToC, 12 agents | v_final: 5324.2 ± 210.8 -> 10576.8 ± 307.4 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on ToC demonstrations · ToC, 12 agents | v_total: 100.01 ± 20.08 -> 342.8 ± 49.36 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on ToC demonstrations · ToC, 12 agents | v_min: 4.60 ± 1.01 -> 44.2 ± $6.4$ | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on ToC demonstrations · ToC, 120 agents | v_final: 3400.5 ± 100.9 -> 8123.4 ± 600.8 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on ToC demonstrations · ToC, 120 agents | v_total: 77.1 ± 14.1 -> 270.0 ± 50.0 | Helped |
| select agents by estimated Exchange Value rather than imitating all data | Behavior cloning on ToC demonstrations · ToC, 120 agents | v_min: 3.51 ± 1.6 -> 33.1 ± 7.1 | Helped |
Surprising findings
- OMAR performed worse than plain BC on StarCraft and in most Overcooked settings, with very low scores on the human dataset (12.5).
Limitations
- Assumes individual agents behave similarly across multiple trajectories.
- Desirability scores are assumed given per collective trajectory and the DVF is only a partial description of desired behavior.
- Experiments are on simulated or small game environments; real-world applicability is untested.
Experimental setup
Datasets: Tragedy of the Commons simulated data Overcooked simulated datasets D^lambda and D^adv Overcooked human anonymized dataset StarCraft Multi-Agent Challenge simulated groups (2s3z, 3s_vs_5z, 6h_vs_8z)
Benchmarks: Overcooked (Cramped Room, Coordination Ring) Tragedy of the Commons StarCraft Multi-Agent Challenge
Metrics: environment desirability value (soups prepared, pool size, t mean estimation error of EVs within-cluster variance of trait lambda
Baselines: BC Group-BC OMAR
Builds on: Shapley Value behavior cloning offline multi-agent reinforcement learning (OMAR) TF-IDF
Closest papers by meaning
- Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors (NeurIPS 2026 · similarity 0.86)
- The Limits of Predicting Agents from Behaviour (ICML 2025 · similarity 0.86)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.85)
- Generalized Intention Modeling in Multi-Agent Reinforcement Learning (NeurIPS 2026 · similarity 0.85)
- Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Games (NeurIPS 2024 · similarity 0.85)
- Multi-Agent Imitation Learning: Value is Easy, Regret is Hard (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.