Research map / Graph learning / GNN architectures
Enhancing Chess Reinforcement Learning with Graph Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AlphaGateau, an AlphaZero-style agent that represents chess positions as graphs and uses a new edge-feature attention GNN layer (GATEAU), showing faster learning than a CNN AlphaZero baseline and transfer from 5x5 to 8x8 chess.
Problem
AlphaZero-style game agents rely on rigid, game-specific grid-based CNN architectures that are computationally expensive to train and cannot easily transfer across board sizes or game variants.
Approach
Represent a chess position as a directed graph with squares as nodes and legal-move-capable edges, and extend the GAT layer with edge features updated through attention (GATEAU) so that policy logits are computed per edge, enabling variable-size inputs and policy outputs.
Key results
- AlphaGateau (5 layers, 1.0M params) reached Elo 2105±42 after 500 iterations vs AlphaZero (5 layers, 2.2M params) at 667±38, with AlphaGateau matching AlphaZero's final Elo after about 10 iterations.
- A 5x5-trained AlphaGateau (10 layers, 1.7M params) reached Elo 807±46 on 8x8 before fine-tuning and 1876±47 after fine-tuning on 8x8.
- On Lichess, the 5-layer AlphaGateau reached bullet 1991, blitz 1829, rapid 1884 Elo.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the 5-layer CNN-based AlphaZero architecture with 5 ResGATEAU layers | AlphaZero on 8×8 chess · 8×8 chess, 500 iterations | Elo: $667\pm38$ -> $2105\pm42$ | Helped |
| Train on 5×5 chess before evaluating on 8×8 chess | AlphaGateau · 8×8 chess, before fine-tuning | Elo | Helped |
| Fine-tune the 5×5-trained model on 8×8 chess | AlphaGateau · 8×8 chess | Elo: $807\pm46$ -> $1876\pm47$ | Helped |
| Generate 1024 self-play games per iteration instead of 256 with a 1M frame window (ablation) | AlphaGateau training · Chess training, frame-window experiment | learning performance | Helped |
| Add a 1M frame window instead of keeping no frame window (ablation) | AlphaGateau training with 256 games per iteration · Chess training, frame-window experiment | learning performance | Helped |
| Use no frame window instead of retaining previous training samples (ablation) | AlphaGateau training with 256 games per iteration · Chess training, frame-window experiment | learning performance | Hurt |
| Increase the number of epochs in one iteration (ablation) | AlphaGateau training · Chess training | learning performance | No clear effect |
| Keep the 1M most recent samples rather than sampling uniformly from the previous frame window (ablation) | AlphaGateau training · Chess training, frame-window selection | learning performance | No clear effect |
Surprising findings
- AlphaGateau trained 13 days 16 hours versus 10 days 3 hours for AlphaZero, yet still learned an order of magnitude faster per iteration.
- The 5x5-only model achieved non-trivial Elo (807) on 8x8 chess without ever seeing an 8x8 position.
Limitations
- Only shallow models (5, 6, 10 residual layers) were tested versus the 40-layer original AlphaZero; no comparison at full depth.
- Each model was trained only once, so confidence intervals reflect only Elo estimation uncertainty, not training variance.
- Evaluated only on chess and small board variants; generality to other games (e.g., Shogi, Risk) is untested.
Experimental setup
Datasets: self-play chess games (8x8 and 5x5 Gardner chess)
Benchmarks: Elo rating via self-play tournaments on 8x8 chess Lichess online play
Metrics: Elo rating 2-sigma confidence interval
Baselines: AlphaZero (CNN, 5 residual layers)
Builds on: AlphaZero Graph Attention Network Gumbel MuZero PGX MCTX
Closest papers by meaning
- Fast and Knowledge-Free Deep Learning for General Game Playing (Student Abstract) (AAAI 2024 · similarity 0.89)
- Amortized Planning with Large-Scale Transformers: A Case Study on Chess (NeurIPS 2024 · similarity 0.87)
- Evidence of Learned Look-Ahead in a Chess-Playing Neural Network (NeurIPS 2024 · similarity 0.86)
- Learning Latent Structures in Network Games via Data-Dependent Gated-Prior Graph Variational Autoencoders (ICML 2024 · similarity 0.86)
- Human-Aligned Chess With a Bit of Search (ICLR 2025 · similarity 0.86)
- The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.