Research map / Reinforcement learning / Visual and representation RL
Thinking Fast and Slow with Deep Learning and Tree Search
Thomas Anthony, Zheng Tian, David Barber
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use tree-policy targets instead of chosen-action targets (ablation) | Hex imitation learning from MCTS · Hex 9×9 | Elo strength: 50±13 Elo stronger | Improved |
| Use tree-policy targets instead of chosen-action targets (ablation) | Hex imitation learning from MCTS · Hex 9×9 | top-1 prediction error: 47.0% -> 47.7% | No clear effect |
| Add 3 batches of 100,000 DAgger moves (ablation) | TPT network · Hex 9×9 | Elo strength: +120 Elo | Improved |
| Add policy-network guidance to MCTS (ablation) | MCTS on Hex · Hex 9×9, versus baseline MCTS | games won: 97% | Improved |
| Double vanilla MCTS iterations instead of using neural policy guidance (ablation) | Vanilla MCTS · Hex 9×9, versus vanilla MCTS | win rate: 56% | Improved |
| Use online dataset aggregation rather than batch ExIt (ablation) | ExIt initialized from the best imitation network · Hex 9×9 | policy strength | Improved |
| Retain the full growing dataset rather than a fixed-size recent-move buffer (ablation) | Online ExIt · Hex 9×9 | policy strength | Improved |
| Add value estimation to the policy network (ablation) | Distributed online ExIt · Hex 9×9 | expert play quality | Improved |
| Use iterative tree-search expert improvement and imitation rather than REINFORCE updates | Hex policy learning · Hex 9×9 | policy strength | Improved |
Closest papers by meaning
- Policy Guided Tree Search for Enhanced LLM Reasoning (ICML 2025 · similarity 0.86)
- Efficient Reinforcement Learning by Discovering Neural Pathways (NeurIPS 2024 · similarity 0.86)
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.86)
- PUZZLES: A Benchmark for Neural Algorithmic Reasoning (NeurIPS 2024 · similarity 0.86)
- In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search (AAAI 2025 · similarity 0.86)
- Mastering Board Games by External and Internal Planning with Language Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
