Research map / Multi-agent systems and games / Cooperative multi-agent RL
Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IMAX-PPO, which adds a multi-agent IQ-Learn-style imitation model that predicts neighboring opponents' next states from local observations and feeds those predictions into MAPPO's actor, evaluated on SMACv2, Google Research Football, and Gold Miner.
Problem
Multi-agent competitive games where allies cannot observe opponents' actions and only see local neighborhood states, making opponent behavior and game dynamics hard to learn, causing slow convergence and instability in MARL.
Approach
Cast opponent behavior prediction as a multi-agent imitation learning problem over next states (not actions), adapt IQ-Learn to state-only local observations, and jointly train this imitator with the allies' MAPPO policy, feeding predicted opponent next states into the actor.
Key results
- On SMACv2 averaged over Protoss, Terran, and Zerg scenarios, IMAX-PPO reaches 62.99%, 61.08%, and 41.48% win rates versus QMIX's 53.56%, 46.95%, 27.99%.
- On Google Research Football, IMAX-PPO averages 98.71% win rate versus MAPPO's 84.39% and QMIX's 9.11%.
- On Gold Miner, IMAX-PPO averages 56.06% win rate versus QMIX's 48.58% and MAPPO's 42.74%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the opponent next-state imitation component to MAPPO | MAPPO, yielding IMAX-PPO · SMACv2 Protoss average | win rate: 53.56% -> 62.99% | Helped |
| Add the opponent next-state imitation component to MAPPO | MAPPO, yielding IMAX-PPO · SMACv2 Terran average | win rate: 46.95% -> 61.08% | Helped |
| Add the opponent next-state imitation component to MAPPO | MAPPO, yielding IMAX-PPO · SMACv2 Zerg average | win rate: 32.71% -> 41.48% | Helped |
| Add the opponent next-state imitation component to MAPPO | MAPPO, yielding IMAX-PPO · GRF average | win rate: 84.39% -> 98.71% | Helped |
| Add the opponent next-state imitation component to MAPPO | MAPPO, yielding IMAX-PPO · Gold Miner average | win rate: 48.58% -> 56.06% | Helped |
Surprising findings
- QMIX outperforms MAPPO on SMACv2 Protoss and Terran in the reported tables, while the paper frames MAPPO as the SOTA baseline.
Limitations
- Assumes enemy policies are fixed during training; if enemies update, the IL process may become unstable and converge slowly.
- Theoretical bounds rely on KL-closeness of ally policies and on bounded Q-values, and the continuous-space results are only stated in order-of-magnitude form.
- Evaluated only on three game environments, with no ablation over training settings such as value normalization or RNN use due to compute limits.
Experimental setup
Datasets: SMACv2 Google Research Football Gold Miner
Benchmarks: SMACv2 Google Research Football Gold Miner
Metrics: win rate
Baselines: MAPPO QMIX
Builds on: IQ-Learn MAPPO PPO SMACv2 soft actor-critic
Closest papers by meaning
- Learning Strategy Representation for Imitation Learning in Multi-Agent Games (AAAI 2025 · similarity 0.90)
- Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.89)
- Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning (ICML 2024 · similarity 0.88)
- Multi-agent cooperation through learning-aware policy gradients (ICLR 2025 · similarity 0.88)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.88)
- Inverse Factorized Soft Q-Learning for Cooperative Multi-agent Imitation Learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.