Research map / Image generation / Autoregressive and latent generation

Simple Hierarchical Planning with Diffusion

ICLR 2024 · Image generation › Autoregressive and latent generation · approach: Diffusion planning and policies · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hierarchical Diffuser, which trains a high-level Sparse Diffuser on every-K-th-state subgoal trajectories and a low-level Diffuser on segments between subgoals, enabling faster and better diffusion-based offline planning.

Problem

Diffuser-style diffusion planners are computationally expensive for long horizons, sample inefficient, and their plans cover the dataset distribution poorly, and they generalize poorly on compositional out-of-distribution tasks.

Approach

Repurpose Diffuser at two levels: a high-level "jumpy" diffuser models subsampled subgoal trajectories (every K-th state, optionally with dense actions) for a larger effective receptive field at lower cost, and a low-level diffuser plans dense segments between adjacent subgoals, with both trained in parallel and low-level plans generated concurrently.

Key results

• HD achieves 139.9 average on single-task Maze2D vs 119.5 for Diffuser and 123.5 for HDMI, and 88.8 average on AntMaze vs 36.0 for Diffuser.
• HD-DA reaches 84.6 average on Gym-MuJoCo vs 77.5 for Diffuser, and 72.5 vs 53.1 average on FrankaKitchen.
• On the compositional OOD task, HD achieves 100% success while all Diffuser kernel-size variants achieve 0%; planning time is about 10x faster than Diffuser on medium and large mazes.

Measured effects reported in the paper

Surprising findings

• Simply enlarging Diffuser's kernel size improves in-distribution performance but hurts compositional OOD generalization, with all Diffuser variants failing the OOD task.
• Including dense actions in the high-level subsampled trajectory improves guidance (return prediction) learning and RL performance in some environments.

Limitations

• Performance depends on dataset quality and coverage as an offline method.
• Fixed subgoal interval K is a task-dependent hyperparameter that may not suit complex real-world scenarios.
• Effectiveness depends on accuracy of the learned value function, which limits how large K can be.

Experimental setup

Datasets: D4RL Maze2D D4RL AntMaze D4RL Gym-MuJoCo D4RL FrankaKitchen Multi2D compositional OOD Maze2D (generated with PD controller)

Benchmarks: D4RL Maze2D D4RL AntMaze Multi2D D4RL Gym-MuJoCo D4RL FrankaKitchen compositional OOD Maze2D

Metrics: normalized return wall-clock training time wall-clock planning time success rate cosine similarity MSE

Baselines: Diffuser MPPI IQL IRIS HiGoC HDMI BC CQL DT TT MOReL BCQ

Builds on: Diffuser classifier guidance HDMI

Closest papers by meaning

• Structural Information-based Hierarchical Diffusion for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Prior-Guided Diffusion Planning for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• State-Covering Trajectory Stitching for Diffusion Planners (NeurIPS 2025 · similarity 0.89)
• Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
• What Makes a Good Diffusion Planner for Decision Making? (ICLR 2025 · similarity 0.89)
• Task-Agnostic Pre-training and Task-Guided Fine-tuning for Versatile Diffusion Planner (ICML 2025 · similarity 0.89)