Research map / Reinforcement learning / Visual and representation RL
Hierarchical Planning and Learning for Robots in Stochastic Settings Using Zero-Shot Option Invention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHARP, which zero-shot invents centroid and interface options from predicted critical regions with dense pseudo-reward option guides, then plans hierarchically and learns option policies with SAC for stochastic robot motion planning.
Problem
Long-horizon, sparse-reward robot motion planning under stochastic dynamics with unknown dynamics, where RL and existing planners scale poorly and hand-coded abstractions are required.
Approach
Zero-shot invention of high-level options (centroid and interface options) from critical-region-based state abstractions predicted by a pretrained UNet, together with zero-shot dense pseudo-reward option guides built from epsilon-clear motion plans, enabling hierarchical planning and refinement in unseen environments without test-time experience.
Key results
- SHARP solves problems about 2x faster than baselines in most cases, with larger gaps in larger environments where baselines solved under 50% of problems SHARP solved.
- SHARP solution reliability is uniformly above 90%, nearly 3x that of RRT-Replan on larger test environments, which had under 33% success in L1-L3.
- Solution costs for SHARP are often orders of magnitude lower in steps than baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHARP's zero-shot option invention and hierarchical planning instead of the baselines | Robot planning in larger environments (L1-L3) · L1-L3 | solution time: 2× | Helped |
| Use SHARP's stochastic policies rather than RRT-Replan | Robot planning in larger environments (L1-L3) · L1-L3 | solution reliability: nearly 3× | Helped |
| Use SHARP's zero-shot options and hierarchical planning rather than the baselines | Robot planning across the evaluation suite · Test environments | solution cost: orders of magnitude | Helped |
| Use SHARP-centroids rather than SHARP-interfaces | SHARP option invention and reuse · Test environments | option reuse rate: 52% vs 45% | Helped |
Surprising findings
- Manipulation environments show a smaller performance gap between approaches due to shorter horizons.
- The RRT-Replan baseline performs below naive re-planning in larger stochastic environments, showing scaling difficulty of existing methods.
Limitations
- Formal completeness guarantees hold only under sufficient conditions (holonomic robots, optimal low-level learner) that the non-holonomic experiments violate.
- Relies on default controllers with unknown support radii and stochasticity simulated by random perturbations in PyBullet.
- Only SAC and RRT-Replan solved any problems within timeouts, so the baseline set is small.
Experimental setup
Datasets: 20 training environments for critical region predictor
Benchmarks: 7 test environments (S1, S2, L1, L2, L3, manipulation M1, M2) with Husky, Limo, and Fetch robots, 60 problems
Metrics: fraction of problems solved vs time average solution cost (steps) solution reliability (success rate)
Baselines: SAC RRT-Replan
Builds on: options framework (Sutton, Precup, Singh 1999) HARP critical-region abstraction (Shah and Srivastava 2022) SAC RRT
Closest papers by meaning
- Learning Neuro-Symbolic Abstractions for Robot Planning and Learning (AAAI 2024 · similarity 0.88)
- Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning (AAAI 2025 · similarity 0.88)
- PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty (NeurIPS 2026 · similarity 0.87)
- Efficient Robotic Policy Learning via Latent Space Backward Planning (ICML 2025 · similarity 0.86)
- Reconciling Spatial and Temporal Abstractions for Goal Representation (ICLR 2024 · similarity 0.86)
- Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstraction (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.