Research map / Multi-agent systems and games / Games and equilibria
Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A short AAAI 2025 invited/position-style summary of the author's research on multi-agent RL theory, covering sample complexity, computational hardness, independent learning, and applications.
Problem
Understanding the statistical and computational foundations of multi-agent reinforcement learning in stochastic games, and why equilibrium is the target solution concept.
Approach
Reviews the author's prior theoretical results: near-optimal model-based sample complexity for two-player zero-sum stochastic games, PPAD-hardness of computing stationary Markov CCE in general-sum stochastic games, and an economics-inspired view of independent learning as the long-run outcome of strategic interaction.
Key results
- Model-based RL for two-player zero-sum stochastic games matches the lower bound up to a |A||B| vs |A|+|B| gap in standard setting (Zhang et al. 2023).
- In the reward-agnostic setting, the lower bound is Omega(|A||B|) and matches the upper bound, so model-based RL is minimax optimal there.
- Computing stationary Markov CCE in general-sum stochastic games is PPAD-hard (Daskalakis, Golowich, Zhang 2023).
Surprising findings
- Computing CCE is tractable for stateless matrix games but PPAD-hard for stochastic games with Markov states, showing multi-agency and sequential decision-making jointly create hardness.
Limitations
- Covers only the author's own prior work; no new experiments or results are presented.
- Theoretical focus on two-player zero-sum and general-sum stochastic games; does not address large-scale deep MARL.
Experimental setup
Builds on: Shapley stochastic games model-based RL
Closest papers by meaning
- Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluation (NeurIPS 2024 · similarity 0.90)
- Decomposing Temporal Equilibrium Strategy for Coordinated Distributed Multi-Agent Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Incentivize without Bonus: Provably Efficient Model-based Online Multi-agent RL for Markov Games (ICML 2025 · similarity 0.89)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.89)
- Exploiting Structure in Offline Multi-Agent RL: The Benefits of Low Interaction Rank (ICLR 2025 · similarity 0.89)
- Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Games (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.