Research map / Reinforcement learning / Visual and representation RL
Efficient Planning with Latent Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LatentDiffuser, which learns a continuous latent action space with a VAE and a latent score-based diffusion prior, and plans via sequence-level energy-guided exact sampling for offline RL.
Problem
Offline RL on temporally extended, sparse-reward tasks is hard; raw-action planning is inefficient and prior latent-action planning (TAP) is limited to discrete latent spaces and needs costly separate planning.
Approach
Jointly learn a continuous latent action space (VAE encoder/decoder) and a latent score-based diffusion prior, then frame planning as energy-guided diffusion sampling in latent space, using a time-dependent energy model trained with a contrastive loss (adapted from QGPO) for exact sequence-level sampling.
Key results
- Gym locomotion average normalized return 87.5 for LatentDiffuser vs 88.0 for D-QL and 86.6 for QGPO.
- Adroit average with expert data 54.6 for LatentDiffuser, best among compared methods (TAP 51.9).
- AntMaze single-task average 78.9 for LatentDiffuser vs 77.1 for HDMI and 73.7 for TAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LatentDiffuser instead of D-QL | Gym locomotion control · Gym locomotion control (Average) | normalized average return: 88.0 -> 87.5 | Hurt |
| Use LatentDiffuser instead of TAP | Adroit · Adroit (Average w/o expert) | normalized average return: 19.6 -> 21.3 | Helped |
| Use LatentDiffuser instead of TAP | Adroit · Adroit (Average w/ expert) | normalized average return: 51.9 -> 54.6 | Helped |
| Use LatentDiffuser instead of HDMI | AntMaze single-task · AntMaze single-task (Single-task Average) | success rate: 77.1 -> 78.9 | Helped |
| Use LatentDiffuser instead of HDMI | AntMaze multi-task · AntMaze multi-task (Multi-task Average) | success rate: 73.6 -> 73.3 | Hurt |
Surprising findings
- On Gym locomotion, LatentDiffuser is only competitive (87.5 average) and does not clearly beat D-QL (88.0), which is a negative result on low-dimensional tasks.
Limitations
- Slow inference due to iterative diffusion sampling.
- Unproven on stochastic dynamics tasks.
- Requires fixed latent step L and planning horizon H.
Experimental setup
Datasets: D4RL Gym locomotion (Med-Expert, Medium, Med-Replay) D4RL Adroit D4RL AntMaze
Benchmarks: D4RL Gym locomotion D4RL Adroit D4RL AntMaze
Metrics: normalized average return
Baselines: CQL IQL D-QL QGPO DT TT MoReL Diffuser DD TAP HDMI CompILE
Builds on: TAP QGPO Diffuser Decision Diffuser Latent diffusion models
Closest papers by meaning
- Reasoning with Latent Diffusion in Offline Reinforcement Learning (ICLR 2024 · similarity 0.90)
- Score Regularized Policy Optimization through Diffusion Behavior (ICLR 2024 · similarity 0.90)
- Prior-Guided Diffusion Planning for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference (NeurIPS 2024 · similarity 0.88)
- State-Covering Trajectory Stitching for Diffusion Planners (NeurIPS 2025 · similarity 0.88)
- Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.