Research map / Reinforcement learning / MDPs and planning
Learning Partial Policies to Speedup MDP Tree Search via Reduction to I.I.D. Learning
Jervis Pinto, Alan Fern
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Q-cost (regret-based) loss instead of zero-one cost for partial policy learning (FT-QCM vs FT-OPI) (ablation) | UCT with learned partial policy · Galcon and Yahtzee | average episodic reward | Improved |
| add forward training to OPI (FT-OPI vs OPI) (ablation) | UCT with learned partial policy · Galcon and Yahtzee | average episodic reward | No clear effect |
| use FT-QCM instead of OPI/FT-OPI at low pruning threshold (ablation) | UCT with learned partial policy · Galcon (sigma_d = 0.5) | average episodic reward | No clear effect |
| use FT-QCM at reduced 50% pruning (ablation) | UCT with learned partial policy · Yahtzee (sigma_d = 0.5) | average episodic reward | Improved |
| use FT-QCM instead of FT-OPI and OPI (ablation) | partial policy pruning · held-out states, Galcon and Yahtzee root | average regret | Improved |
| use FT-QCM instead of FT-OPI and OPI (ablation) | partial policy pruning · held-out states, Galcon and Yahtzee root | pruning error | Improved |
| go deeper in the tree (depth d>0) for learned policies | FT-QCM · Galcon | regret | Hurt |
| go deeper in the tree (depth d>0) for learned policies | FT-QCM · Yahtzee | regret | No clear effect |
| prune too many actions (large sigma_d) | UCT with partial policy · Galcon and Yahtzee | search performance | Hurt |
| prune too few actions | UCT with partial policy · Galcon and Yahtzee | search performance | Hurt |
| use FT-QCM pruning with small search budget | UCT(FT-QCM) · Galcon and Yahtzee | average episodic reward | Improved |
Closest papers by meaning
- SPOT: Scalable Policy Optimization with Trees for Markov Decision Processes (NeurIPS 2025 · similarity 0.90)
- A Surprisingly Simple Continuous-Action POMDP Solver: Lazy Cross-Entropy Search Over Policy Trees (AAAI 2024 · similarity 0.88)
- Tree Search-Based Policy Optimization under Stochastic Execution Delay (ICLR 2024 · similarity 0.87)
- Subgoal-Guided Policy Heuristic Search with Learned Subgoals (ICML 2025 · similarity 0.87)
- RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation (NeurIPS 2024 · similarity 0.87)
- In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
