Research map / Multi-agent systems and games / Games and equilibria
A Sharp Analysis of Model-based Reinforcement Learning with Self-Play
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the auxiliary bonus γ to value iteration | Nash-VI · Two-player zero-sum Markov games | sample complexity (episodes): \tlO(H^4S^2AB/\epsilon^2) -> \tlO(H^4SAB/\epsilon^2) | Helped |
| Replace the Hoeffding-type bonus with a Bernstein-type bonus (ablation) | Nash-VI · Two-player zero-sum Markov games | sample complexity (episodes): \tlO(H^4SAB/\epsilon^2) -> \tlO(H^3SAB/\epsilon^2) | Helped |
| Use a reward-free exploration phase and retain only an upper bound on the value | VI-Zero · Zero-sum Markov games with N known reward tasks | sample complexity (episodes): \tlO(H^3SAB/\epsilon^2) -> \tlO(H^4SAB\log N/\epsilon^2) | Mixed |
| Use a maximizing exploration policy rather than a CCE policy (ablation) | VI-Zero · Reward-free exploration in zero-sum Markov games | sample complexity (episodes) | No clear effect |
| Use a larger exploration bonus, by a √S factor (ablation) | Multi-VI-Zero · Multi-player general-sum Markov games | sample complexity (episodes): H^4S^2 | Hurt |
Closest papers by meaning
- Provable Memory Efficient Self-Play Algorithm for Model-free Reinforcement Learning (ICLR 2024 · similarity 0.94)
- Improving Sample Efficiency of Model-Free Algorithms for Zero-Sum Markov Games (ICML 2024 · similarity 0.94)
- Incentivize without Bonus: Provably Efficient Model-based Online Multi-agent RL for Markov Games (ICML 2025 · similarity 0.91)
- Optimistic Policy Gradient in Multi-Player Markov Games with a Single Controller: Convergence beyond the Minty Property (AAAI 2024 · similarity 0.90)
- Independent Learning of Nash Equilibria in Partially Observable Markov Potential Games with Decoupled Dynamics (NeurIPS 2026 · similarity 0.90)
- Sample-Efficient Multi-Agent RL: An Optimization Perspective (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.