Research map / Reinforcement learning / MDPs and planning
Planning with Abstract Learned Models While Learning Transferable Subtasks
John Winder, Stephanie Milani, Matthew Landen, Erebus Oh, Shane Parr, Shawn Squire, Marie desJardins, Cynthia Matuszek
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PALM with an expert Taxi hierarchy instead of R-MAXQ using the same hierarchy | PALM-ET vs. R-MAXQ · Deterministic small Taxi | performance | Hurt |
| Use PALM rather than R-MAXQ | PALM vs. R-MAXQ · Stochastic Taxi | learning efficiency | Improved |
| Use PALM rather than R-MAXQ with the expert Cleanup hierarchy | PALM-EC vs. R-MAXQ · Small Cleanup | runtime per episode: 100 -> 10 ms; R-MAXQ: 10^4 ms | Improved |
| Learn the Taxi hierarchy from demonstrations rather than use the expert hierarchy | PALM-HT vs. PALM-ET · Taxi | samples to reach asymptotic performance | Improved |
| Learn the Taxi hierarchy from demonstrations rather than use the expert hierarchy | PALM-HT vs. PALM-ET · Taxi with multiple passengers | cumulative reward | Hurt |
| Use the learned Cleanup hierarchy instead of the expert Cleanup hierarchy | PALM-HC vs. PALM-EC · Cleanup | samples to convergence | Improved |
| Amend the Cleanup hierarchy using expert knowledge after observing the learned hierarchy | PALM-AC vs. PALM-HC · Cleanup | samples to convergence | Improved |
| Use the original expert Cleanup hierarchy rather than model-free Q-learning | PALM-EC vs. QL · Cleanup | steps to learn | Hurt |
| Transfer a converged Navigation subtask model to a new task set | PALM-ET given Nav vs. PALM-ET · New related tasks | cumulative reward | Improved |
| Train HierGen on sufficiently complex Cleanup demonstrations with no shared features | PALM-HC · Cleanup | hierarchy structure | Hurt |
Closest papers by meaning
- Learning Planning Abstractions from Language (ICLR 2024 · similarity 0.88)
- Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning (AAAI 2025 · similarity 0.88)
- Learning Neuro-Symbolic Abstractions for Robot Planning and Learning (AAAI 2024 · similarity 0.87)
- Reconciling Spatial and Temporal Abstractions for Goal Representation (ICLR 2024 · similarity 0.86)
- Reward-Respecting Subtasks for Model-Based Reinforcement Learning (Abstract Reprint) (AAAI 2024 · similarity 0.86)
- Hierarchical Planning and Learning for Robots in Stochastic Settings Using Zero-Shot Option Invention (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
