Research map / Multi-agent systems and games / Games and equilibria
Oracles & Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a contextual Meta-RL follower oracle instead of PPO+Q-learning followers | PPO leader on iterated matrix games · 12 canonical symmetric iterated matrix games | sample efficiency: around 50k environment steps; around 500k environment steps | Helped |
| Use contextual Meta-RL follower oracle rather than Good Shepherd ES-MD | Stackelberg learning on iterated matrix games · 12 canonical symmetric iterated matrix games | final mean episode reward | No clear effect |
| Use contextual Meta-RL follower oracle | PPO leader in bilateral trade · Atari 2600 bilateral trade | Stackelberg equilibrium behavior | Helped |
| Hide follower-oracle queries from the leader instead of including them in the leader episode (ablation) | Contextual-policy Stackelberg approach · Iterated Prisoner's Dilemma | learning sensible behavior | Hurt |
| Allow the leader to behave differently during oracle queries and real play (ablation) | Contextual-policy Stackelberg approach · Iterated Prisoner's Dilemma | Stackelberg equilibrium validity | Hurt |
| Give the leader reward during follower learning (ablation) | Leader learning with a Q-learning follower · Battle of the Sexes | learning the Stackelberg equilibrium: 2.0 | Hurt |
| Carry the follower Q-table across leader updates instead of resetting it (ablation) | Leader learning with a Q-learning follower · Battle of the Sexes | convergence to the Stackelberg equilibrium | Hurt |
| Have followers immediately best-respond without query steps folded into the leader's experience (ablation) | Tabular Q-learning or policy-gradient leader · Constructed stochastic Markov game | learning the optimal leader policy | Hurt |
Closest papers by meaning
- Stackelberg Actor-Critic: Game-Theoretic Reinforcement Learning Algorithms (AAAI 2022 · similarity 0.90)
- Sample-Efficient Learning of Stackelberg Equilibria in General-Sum Games (NeurIPS 2021 · similarity 0.88)
- Sequential Asynchronous Action Coordination in Multi-Agent Systems: A Stackelberg Decision Transformer Approach (ICML 2024 · similarity 0.88)
- Learning in Stackelberg Mean Field Games: A Non-Asymptotic Analysis (NeurIPS 2025 · similarity 0.88)
- Transition-Informed Reinforcement Learning for Large-Scale Stackelberg Mean-Field Games (AAAI 2024 · similarity 0.87)
- Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.