Research map / Reinforcement learning / MDPs and planning

Hitting Time Isomorphism for Multi-Stage Planning with Foundation Policies

NeurIPS 2026 · Reinforcement learning › MDPs and planning · approach: Rich-observation exploration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Isomorphic Embedding Learning (IEL), an operator-theoretic hitting-time representation for offline goal-agnostic foundation policies, with a directed distance and graph-based planning, evaluated on six D4RL maze/kitchen goal-conditioned tasks against HILP.

Problem

Offline foundation policy learning needs a goal-agnostic representation whose distances are directional and satisfy the triangle inequality so that multi-stage planning can compose subgoals; existing symmetric hitting-time or successor methods fail one or more of these.

Approach

Learns a Hilbert-space displacement geometry in which expected hitting times are linear functionals of latent displacements (phi(g)-phi(x)), combining a HILP-style consistency objective with explicit hitting-time regression and an angular asymmetry factor exp(beta(1-cos)), then plans with shortest paths over a graph of latent states.

Key results

• On antmaze-ultra-diverse, IEL Asym-Graph reaches 79.0 ± 4.5 normalized return versus 39.8 ± 22.1 for HILP Rec-Mid.
• IEL Asym-Graph achieves the best score on four of six datasets (e.g., 71.8 on antmaze-large-diverse vs 68.0 for HILP Rec-Mid), with symmetric graph gains on five environments over HILP's midpoint planning.
• Asymmetric planning causes minor degradation on two environments (kitchen-mixed 57.2 vs 57.8 symmetric; kitchen-partial tied).

Surprising findings

• Asymmetric graph planning causes minor degradation on two of six environments, and in some cases the symmetric variant beats it.

Limitations

• Range condition and identifiability rely on linear closure assumptions that are not verified in nonlinear environments.
• Global hitting-time error scales as 1/(1-rho), sensitive to transient spectral radius.
• The TD upper bound used to link to HILP may need tightening for stronger directionality.

Experimental setup

Datasets: D4RL antmaze-large-diverse D4RL antmaze-large-play D4RL antmaze-ultra-diverse D4RL antmaze-ultra-play D4RL kitchen-partial D4RL kitchen-mixed

Benchmarks: D4RL antmaze-large-diverse D4RL antmaze-large-play D4RL antmaze-ultra-diverse D4RL antmaze-ultra-play D4RL kitchen-partial D4RL kitchen-mixed

Metrics: normalized return

Baselines: HILP GC-IQL GC-CQL

Builds on: HILP IQL Koopman operator TMD GAS

Closest papers by meaning

• Structural Information-based Hierarchical Diffusion for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
• Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning (ICML 2025 · similarity 0.87)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models (NeurIPS 2025 · similarity 0.87)
• Efficient Planning with Latent Diffusion (ICLR 2024 · similarity 0.87)
• State-Covering Trajectory Stitching for Diffusion Planners (NeurIPS 2025 · similarity 0.87)
• Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning (ICML 2025 · similarity 0.87)