Research map / Reinforcement learning / Visual and representation RL
Regret-Guided Search Control for Efficient Learning in AlphaZero
Yun-Jui Tsai, Wei-Yu Chen, Yan-Ru Ju, Yu-Hung Chang, Ti-Rong Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add regret-guided search control (RGSC) to AlphaZero | AlphaZero · 9x9 Go vs KataGo | win rate: 45.5% -> 53.6% | Improved |
| add RGSC to AlphaZero | AlphaZero · 10x10 Othello vs Ludii alpha-beta | win rate: 51.7% -> 57.8% | Improved |
| add RGSC to AlphaZero | AlphaZero · 11x11 Hex vs MoHex | win rate: 83.6% -> 91.1% | Improved |
| replace uniform state sampling (Go-Exploit) with regret-prioritized sampling | Go-Exploit · 9x9 Go vs KataGo | win rate: 49.5% -> 53.6% | Improved |
| add RGSC when continuing training of a well-trained model | AlphaZero continued training · 9x9 Go vs KataGo (well-trained) | win rate: 70.2% -> 78.2% | Improved |
| continue training with Go-Exploit instead of AlphaZero | well-trained 9x9 Go model · 9x9 Go (well-trained) | Elo: Go-Exploit performs worse than AlphaZero by 45 Elo | Hurt |
| use regret ranking network instead of regret value network for state selection (ablation) | RGSC · 9x9 Go, 10x10 Othello, 11x11 Hex | playing performance | Improved |
| rank states with regret ranking network instead of value network (ablation) | RGSC · Top 2,000 selected states | average true regret | Improved |
| replay high-regret states from prioritized regret buffer | RGSC · 9x9 Go | average regret: 0.655 -> 0.296 | Improved |
| raise buffer sampling probability lambda from 0.5 to 0.9 (ablation) | RGSC · 11x11 Hex | training stability | Hurt |
| lower buffer sampling temperature tau from 1 to 0.1 (ablation) | RGSC · three games | playing performance | Improved |
| increase buffer size from 100 to 500 or 1000 (ablation) | RGSC · three games | playing performance | No clear effect |
Closest papers by meaning
- A Minimaximalist Approach to Reinforcement Learning from Human Feedback (ICML 2024 · similarity 0.88)
- Refining Minimax Regret for Unsupervised Environment Design (ICML 2024 · similarity 0.88)
- Search Self-Play: Pushing the Frontier of Agent Capability without Supervision (ICLR 2026 · similarity 0.88)
- Efficient Preference-Based Reinforcement Learning: Randomized Exploration meets Experimental Design (NeurIPS 2025 · similarity 0.88)
- Query-Policy Misalignment in Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.88)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
