Research map / Large language models / Agents, code and math
Mastering Board Games by External and Internal Planning with Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pre-trains multi-action-value Transformers (MAV, 2.7B and 1B) on textual board game data for chess, Chess960, Connect Four and Hex, then uses them for external MCTS planning and distills search into the model for single-call internal search.
Problem
LLMs struggle to play board games reliably because they cannot consistently reason over possible futures with a world model; the paper asks how internal (in-context) and external (search-based) planning can be trained to improve game-playing strength.
Approach
A single Transformer (MAV) jointly acts as world model (state tracking, legal moves, terminal detection), value function (bucketed win-probability per move) and policy, enabling AlphaZero-style MCTS without a game engine (dynamic virtual counts, async MCTS) and distilling minimax search traces into the model for single-call internal search.
Key results
- MAV-2.7B with mean scoring reaches external Elo 2923 in chess, outperforming Stockfish-L10 (2788) and performing comparably to the UAI model.
- External Async MCTS with 100 simulations (MCTS over MAV) reaches internal Elo 68 above searchless MAV-2.7B in chess, with gains growing logarithmically with simulations; the abstract claims about +300 Elo gains overall.
- Internal search fine-tuned model builds accurately shaped trees 99.6% of the time and its playing strength increases with token budget.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use mean scoring instead of max scoring (ablation) | MAV-large · Chess | external Elo: 2875 -> 2923 | Helped |
| add 100 simulations of MAV-guided external MCTS | MAV-large · Chess | internal Elo: +68 Elo | Helped |
| increase the number of external MCTS simulations | MAV-guided MCTS · Chess, Chess960, Connect Four, and Hex | Elo | Helped |
| add MAV-guided external MCTS | MAV · Connect Four | internal Elo: at least 244 Elo | Helped |
| use MAV-guided external MCTS instead of basic MCTS | MAV-guided MCTS · Hex | internal Elo | Helped |
| increase the internal search budget | MAVIS · Chess | external Elo | Helped |
| use a tuned dynamic virtual count instead of static virtual counts (ablation) | Async MCTS · Head-to-head comparisons | performance | Helped |
| reduce the Async MCTS batch size (ablation) | Async MCTS · Head-to-head comparisons | performance | Helped |
Surprising findings
- Only about 10% of positions played by MAV-2.7B in evaluation appear in training data, and between moves 20 and 50 virtually none do, indicating generalization rather than memorization.
- Hex and Connect Four results differ notably: Connect Four gains from MCTS over MAV are relatively small, whereas Hex shows a large gain from MAV over basic MCTS.
Limitations
- Trained exclusively on game data, so no natural-language communication ability.
- Relies on game engines (Stockfish, neurobenzene, Fhourstones) for annotation and requires large data generation.
- Hex final model lacked state-tracking, so the engine was used for legal moves and transitions.
Experimental setup
Datasets: Lichess rated games Lichess Chess960 games TCEC games Hex self-play (neurobenzene) Connect Four self-play (Fhourstones) Stockfish 16 annotations
Benchmarks: Internal and external Elo tournaments in chess, Chess960, Connect Four, Hex Lichess puzzles and OOD positions legality/state-tracking accuracy
Metrics: Elo rating legal move rate top_all precision/recall/F1 FEN accuracy win rate
Baselines: Stockfish (levels L0-L20) UAI (ChessBot / Ruoss et al. 270M chess transformer) basic MCTS with random rollouts MAV without search
Builds on: AlphaZero MuZero Stream of Search Gemini architecture Tree of Thoughts
Closest papers by meaning
- Implicit Search via Discrete Diffusion: A Study on Chess (ICLR 2025 · similarity 0.92)
- Can Large Language Models Master Complex Card Games? (NeurIPS 2025 · similarity 0.90)
- Amortized Planning with Large-Scale Transformers: A Case Study on Chess (NeurIPS 2024 · similarity 0.89)
- GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations (NeurIPS 2024 · similarity 0.89)
- GAVEL: Generating Games via Evolution and Language Models (NeurIPS 2024 · similarity 0.89)
- PokerBench: Training Large Language Models to Become Professional Poker Players (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.