Research map / Reinforcement learning / Visual and representation RL
Beyond Policy Training: Recursive Solution Search from Unannotated Videos
Lipeng Wan, Jianhui Gu, Junjie Ma, Anbang Wang, Xuguang Lan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the 50-sample warm-up set with coarse stage labels (ablation) | PFR-Search backward planning · Minecraft (5 tasks average) | success rate (%): 24.8 -> 52.0 | Improved |
| increase unannotated videos from 300 to 600 to 1200 clips (ablation) | PFR-Search backward planning · Minecraft (5 tasks average) | success rate (%): 24.0 -> 43.4 -> 52.0 | Improved |
| add in-flow consistency (ablation) | PFR-Search backward planning · Minecraft (5 tasks average) | success rate (%): 50.8 -> 52.0 | Improved |
| add cross-flow consistency (ablation) | PFR-Search backward planning · Minecraft (5 tasks average) | success rate (%): 49.2 -> 52.0 | Improved |
| add both in-flow and cross-flow consistency (ablation) | PFR-Search backward planning · Minecraft (5 tasks average) | success rate (%): 48.8 -> 52.0 | Improved |
| add cross-flow consistency (ablation) | PFR-Search backward planning · Minecraft Harvest log | success rate (%): 63.0 -> 62.0 | Hurt |
| add in-flow consistency (ablation) | PFR-Search end-to-end search · Minecraft (5 tasks average) | success rate (%): 15.0 -> 16.0 | Improved |
| add cross-flow consistency (ablation) | PFR-Search end-to-end search · Minecraft (5 tasks average) | success rate (%): 14.4 -> 16.0 | Improved |
| add both consistencies (ablation) | PFR-Search end-to-end search · Minecraft (5 tasks average) | success rate (%): 14.2 -> 16.0 | Improved |
| add task-flow guidance and replay over random exploration | random exploration · Minecraft (5 tasks average) | forward search success rate (%): 1.8 -> 16.0 | Improved |
| replace learned task flows with oracle task flows from human demonstrations | PFR-Search end-to-end · Minecraft (5 tasks average) | forward search success rate (%): 16.0 -> 27.8 | Improved |
| increase the search budget | PFR-Search vs random exploration · Minecraft (two tasks) | success rate: PFR-Search improves steadily; random only marginally | Mixed |
Closest papers by meaning
- Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search (ICLR 2019 · similarity 0.88)
- Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs (NeurIPS 2025 · similarity 0.87)
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.86)
- Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.86)
- PWM: Policy Learning with Multi-Task World Models (ICLR 2025 · similarity 0.86)
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
