Research map / Multi-agent systems and games / Games and equilibria
Fast Peer Adaptation with Context-aware Exploration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PACE, which trains a context-aware ego policy with a peer-identification auxiliary task whose posterior probability serves as an exploration reward, enabling fast adaptation to unknown peers in partially observable games.
Problem
Fast adaptation to unknown peers (partners or opponents) in partially observable, long-horizon multi-agent games, where agents must explore to reveal peer strategies before exploiting them across multiple episodes.
Approach
A peer identification auxiliary task trains an identifier on the context encoder; the posterior probability of the true peer identities is added as an intrinsic exploration reward (a mutual-information objective), jointly with a cross-episode RL objective, so the policy learns to explore when uncertain and exploit when confident.
Key results
- PACE achieves 0.553 success rate in PO-Overcooked versus at most 0.111 for baselines (Generalist 0.111, LIAMX 0.099, GSCU 0.100).
- In Kuhn Poker PACE reaches 0.047 average reward vs 0.027 for the best baseline (LIAMX), approaching Oracle 0.077.
- In Predator-Prey-W PACE reaches -3.93 reward vs -4.34 for the best baseline (LIAMX), with Oracle at -2.82.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PACE instead of the strongest baseline, LIAMX | Peer adaptation · Kuhn Poker | Reward: 0.027 $\pm$ 0.008 -> 0.047 $\pm$ 0.004 | Helped |
| Use PACE instead of the strongest baseline, Generalist | Peer adaptation · PO-Overcooked | Success rate: 0.111 $\pm$ 0.016 -> 0.553 $\pm$ 0.029 | Helped |
| Use PACE instead of the strongest baseline, LIAMX | Peer adaptation · Predator-Prey-W | Reward: -4.34 $\pm$ 0.08 -> -3.93 $\pm$ 0.04 | Helped |
| Switch the peer after 5 episodes during 10-episode adaptation | PACE in PO-Overcooked · PO-Overcooked | Success rate: 0.553 $\pm$ 0.029 -> 0.435 $\pm$ 0.027 | Hurt |
| Add the exploration reward (ablation) | PACE-reward in PO-Overcooked · PO-Overcooked | Success rate: 0.173 $\pm$ 0.016 -> 0.553 $\pm$ 0.029 | Helped |
| Add the auxiliary task (ablation) | PACE-reward-aux in PO-Overcooked · PO-Overcooked | Success rate: 0.143 $\pm$ 0.020 -> 0.173 $\pm$ 0.016 | Helped |
| Use discount factor γ=0.9 instead of γ=0.99 (ablation) | PACE · PO-Overcooked | Success rate | Hurt |
| Use discount factor γ=0.999 instead of γ=0.99 (ablation) | PACE · PO-Overcooked | Training stability | Hurt |
Surprising findings
- Removing the exploration reward drops PO-Overcooked success from 0.553 to 0.173, and further removing the auxiliary task drops it to 0.143.
- GSCU, which uses post-episode peer observations, performs worst in Kuhn Poker (-0.041) despite adapting over time.
Limitations
- Requires a diverse training peer pool, and performance depends on the quality of that pool.
- Experiments keep the same number of agents at test time as in training.
- No human peers are used in evaluation.
Experimental setup
Datasets: Kuhn Poker PO-Overcooked Predator-Prey-W
Benchmarks: Kuhn Poker PO-Overcooked Predator-Prey-W
Metrics: average episodic reward success rate
Baselines: GSCU LIAM LIAMX LILI Generalist
Builds on: PPO LIAM LILI GSCU
Closest papers by meaning
- Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations (AAAI 2024 · similarity 0.90)
- Multi-agent cooperation through learning-aware policy gradients (ICLR 2025 · similarity 0.88)
- Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.87)
- Adaptively Coordinating with Novel Partners via Learned Latent Strategies (NeurIPS 2025 · similarity 0.86)
- Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Games (NeurIPS 2024 · similarity 0.86)
- Opponent Modeling based on Subgoal Inference (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.