Research map / Reinforcement learning / Policy gradient and actor-critic
Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
Kazuki Ota, Takayuki Osa, Motoki Omura, Tatsuya Harada
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| KLENT instead of Gumbel AlphaZero for learning efficiency | Gumbel AlphaZero · Five board games average | simulator evaluations to 50% win rate: 300 million -> 75 million | Improved |
| KLENT with test-time MCTS versus Gumbel AlphaZero | Gumbel AlphaZero · Five board games average (test-time MCTS) | win rate (%): 53.6% -> 77.2% | Improved |
| KLENT versus Gumbel AlphaZero on 9x9 Go | Gumbel AlphaZero · 9x9 Go (test-time MCTS) | win rate (%): 37% -> 89% | Improved |
| KLENT versus Gumbel AlphaZero on Hex | Gumbel AlphaZero · Hex (test-time MCTS) | win rate (%): 47% -> 98% | Improved |
| KLENT versus Gumbel AlphaZero on Animal Shogi | Gumbel AlphaZero · Animal Shogi (test-time MCTS) | win rate (%): 67% -> 63% | Hurt |
| KLENT versus Gumbel AlphaZero on Othello | Gumbel AlphaZero · Othello (test-time MCTS) | win rate (%): 47% -> 55% | Improved |
| add reverse-KL regularization (vs entropy only) (ablation) | KLENT · Five board games | win rate learning curve | Improved |
| add entropy regularization (vs KL only) (ablation) | KLENT · Five board games | win rate learning curve | Improved |
| use lambda-returns instead of 1-step returns (ablation) | KLENT · 9x9 Go and Hex | win rate learning curve | Improved |
| use lambda-returns instead of Monte Carlo returns (ablation) | KLENT · 9x9 Go and Hex | win rate learning curve | Improved |
| KLENT versus AlphaZero on 19x19 Go | AlphaZero · 19x19 Go | learning curve | No clear effect |
| KLENT versus AlphaZero on Gardner Chess | AlphaZero · Gardner Chess (test-time MCTS) | win rate (%): 64% -> 81% | Improved |
Closest papers by meaning
- Fast Policy Extragradient Methods for Competitive Games with Entropy Regularization (NeurIPS 2021 · similarity 0.91)
- Policy Gradient Methods Converge Globally in Imperfect-Information Extensive-Form Games (NeurIPS 2025 · similarity 0.90)
- Policy Optimization for Markov Games: Unified Framework and Faster Convergence (NeurIPS 2022 · similarity 0.90)
- Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach (NeurIPS 2025 · similarity 0.89)
- A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence (ICLR 2025 · similarity 0.89)
- Learning Imperfect Information Extensive-form Games with Last-iterate Convergence under Bandit Feedback (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
