Research map / Reinforcement learning / Visual and representation RL
On the role of planning in model-based deep reinforcement learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use deeper search for policy learning instead of One-Step targets (ablation) | MuZero · Across environments | normalized reward | Helped |
| Use search to select training actions instead of only the policy prior (ablation) | MuZero · Across environments | normalized reward | Helped |
| Use search for both policy learning and training action selection (ablation) | MuZero · Across environments | normalized reward | Helped |
| Use search at evaluation in addition to learning and training action selection (ablation) | MuZero · Across environments | normalized reward: 100% | Mixed |
| Limit tree search depth to 2 rather than using deep search (ablation) | MuZero · Sokoban; 9x9 Go | normalized reward | No clear effect |
| Use deeper trees rather than shallow trees (ablation) | MuZero · Minipacman | normalized reward | Hurt |
| Use deeper trees rather than shallow trees (ablation) | MuZero · Ms. Pacman; Acrobot; Sokoban; 9x9 Go | normalized reward | Helped |
| Increase pUCT exploration depth beyond the root (ablation) | MuZero · All environments except 9x9 Go | normalized reward | No clear effect |
| Increase pUCT exploration depth beyond the root (ablation) | MuZero · 9x9 Go | normalized reward | Helped |
| Increase the number of search simulations during training (ablation) | MuZero · All environments | normalized reward | Mixed |
| Use more than two search simulations during training (ablation) | MuZero · Ms. Pacman | normalized reward | No clear effect |
| Increase search at evaluation using the learned model (ablation) | MuZero · Acrobot; Sokoban | reward | Helped |
Closest papers by meaning
- Improving planning and MBRL with temporally-extended actions (NeurIPS 2025 · similarity 0.89)
- Efficient Multi-agent Reinforcement Learning by Planning (ICLR 2024 · similarity 0.89)
- Towards General-Purpose Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Interpreting Emergent Planning in Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.88)
- OptionZero: Planning with Learned Options (ICLR 2025 · similarity 0.88)
- Policy-shaped prediction: avoiding distractions in model-based reinforcement learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.