AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

Efficient Learning for AlphaZero via Path Consistency

Dengwei Zhao, Shikui Tu, Lei Xu

ICML 2022 · Reinforcement learning › Visual and representation RL · approach: Scalable RL systems and environments

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add path consistency loss (lambda) to AlphaZero trainingPCZero online learning · 13x13 Hex vs MoHex 2.0winning rate: 84.3% to 93.5% (beta=0)Improved
add feature consistency loss (beta) on top of value PC (ablation)PCZero online learning · 13x13 Hex vs MoHex 2.0winning rate: 93.5% to 94.1%Improved
add PC loss to AlphaZero trainingonline learning on Othello · Othello vs 3-ply Edaxwinning rate: 69.5% to 74.8%Improved
add PC loss to AlphaZero trainingoffline learning on 13x13 Hex · MCTS player vs offline AlphaZerowinning rate: 61.5% (MCTS), 52.1% (greedy)Improved
add PC loss to AlphaZero trainingoffline learning on Othello and Gomoku · MCTS player vs offline AlphaZerowinning rate: Othello 80.5%, Gomoku 64.0%Improved
add PC loss to offline AlphaZero for greedy playersoffline learning on Hex 8x8, 9x9, 13x13, Othello, Gomoku · greedy player vs offline AlphaZerowinning rate: 50.5% to 53.1%No clear effect
add feature consistency loss alone (lambda=0) to offline AlphaZero (ablation)offline 13x13 Hex · MCTS player vs offline AlphaZerowinning rate: 50.3%No clear effect
add feature consistency (beta=1.0) on top of PC loss (ablation)offline 13x13 Hex · MCTS player vs offline AlphaZerowinning rate: 61.5% to 70.7%Improved
increase value loss weight gamma to 2.0 or 3.0 instead of PC loss (ablation)offline AlphaZero on 13x13 Hex · MCTS player vs offline AlphaZero (gamma=1.0)winning rate: 45.9% (gamma=2.0), 55.0% (gamma=3.0)No clear effect
include MCTS heuristic path (scouted by search) in PC window alongside historical path (ablation)8x8 Hex MCTS-PCZero · Elo / tournament vs PCZerowinning rate: 50.0% vs final PCZero; converges faster to higher EloImproved
use only heuristic path (k=5, l=0) vs only historical path (l=5, k=0) (ablation)8x8 Hex MCTS-PCZero window · tournamentwinning rate: 57.0% (heuristic-only vs historical-only)Mixed
unbounded heuristic path length (k=inf) vs k=5 in MCTS-PCZero (ablation)8x8 and 13x13 Hex MCTS-PCZero · tournamentwinning rate: 50.8% vs MCTS-PCZero (l=k=5) on 8x8; 53.0% on 13x13 vs l=k=5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy