Research map / Reinforcement learning / Visual and representation RL
Reconciling Spatial and Temporal Abstractions for Goal Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STAR, a three-layer feudal HRL algorithm that combines reachability-aware spatial goal abstraction (learned online via neural reachability analysis) with a middle-level temporal tutor agent, and gives a suboptimality bound.
Problem
Goal representations for hierarchical RL that preserve temporally abstract dynamics do not scale when transition relations depend on many variables, and spatial abstraction approaches (GARA) struggle because low-level agents must reach distant abstract sets in high-dimensional continuous control.
Approach
Combine a discrete reachability-aware spatial abstraction (Commander selects abstract goal sets) with a middle-level Tutor that selects intermediate subgoals reachable by the Controller, giving both spatial and temporal abstraction in a three-layer hierarchy, plus a theoretical suboptimality bound.
Key results
- Theoretical result: a suboptimality bound on the hierarchical policy learned under the reachability-aware set-based abstraction, with refinements improving the abstraction.
- Ant Fall analysis shows the Commander refines goals above the pit along the z-axis and identifies a bridge-like region by 3M timesteps.
Limitations
- Reachability analysis requires neural network verification tools (Ai2) and hyperrectangle abstractions, which add cost.
- Non-stationarity across three levels requires heuristics such as stability checks before refinement.
Experimental setup
Datasets: Ant Maze Ant Fall Ant Maze Cam
Benchmarks: Ant Maze Ant Fall Ant Maze Cam
Metrics: success rate
Baselines: GARA HRAC HIRO LESSON
Builds on: GARA HRAC HIRO TD3
Closest papers by meaning
- Hierarchical Goal Abstractions via Learned Subset Relations (ICML 2026 · similarity 0.91)
- Sample Complexity of Goal-Conditioned Hierarchical Reinforcement Learning (NeurIPS 2023 · similarity 0.90)
- Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning (AAAI 2025 · similarity 0.89)
- PcLast: Discovering Plannable Continuous Latent States (ICML 2024 · similarity 0.88)
- Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.