Research map / Probabilistic and scientific modeling / Variational and generative sampling
Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Latent Plan Transformer, which models trajectories and final returns through a latent plan variable trained by MLE with Langevin posterior sampling, and infers the plan from a target return at test time for planning.
Problem
Planning and decision-making from offline trajectory-return pairs without step-wise rewards, where temporal consistency and long-horizon credit assignment are hard for return-conditioned sequence models.
Approach
A latent plan variable z (a UNet transform of Gaussian noise) connects a causal Transformer trajectory generator and a return predictor; training uses MLE with posterior sampling of z via Langevin dynamics and planning is done by sampling z given a desired return, replacing return-to-go conditioning.
Key results
- LPT-EI reaches 92.4±0.80 on Antmaze-umaze versus 53.3 for DT, and 89.93±0.61 on hopper-medium-replay with final-return supervision.
- On Maze2D-large, LPT-EI scores 45.89±2.98 versus 31.0 for QDT and 2.3 for DT.
- On Connect Four, LPT reaches 0.99±0.01, matching ESPER (0.99±0.03) and above DT (0.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use exploitation-inclined inference (LPT-EI) instead of standard LPT inference | LPT · Gym-Mujoco, Franka Kitchen, Maze2D, and Antmaze | normalized score: 43.13\pm0.38 -> 43.53\pm0.08; 39.64\pm0.83 -> 40.66\pm 0.12; 58.52\pm1.92 -> 63.83\pm 1.47; 82.29\pm1.26 -> 89.93 \pm 0. | Helped |
| Replace step-wise-reward inputs with trajectory-return pairs and use LPT | LPT · Gym-Mujoco, Franka Kitchen, Maze2D, and Antmaze | normalized score | Mixed |
| Use LPT for sparse-reward Maze2D planning rather than QDT | LPT · Maze2D-umaze, Maze2D-medium, and Maze2D-large | normalized score: 57.3\pm8.2 -> 65.43\pm2.91; 13.3\pm5.6 -> 20.62\pm1.81; 31.0\pm19.8 -> 37.21\pm2.05 | Helped |
| Use LPT rather than CQL | LPT · Antmaze-umaze | normalized score: 74.0 -> 80.8\pm4.83 | Helped |
| Remove the UNet learned prior (ablation) | LPT · halfcheetah-medium-replay | normalized score: 39.64\pm0.83 -> 34.70\pm1.58 | Hurt |
| Remove the UNet learned prior (ablation) | LPT · walker2d-medium-replay | normalized score: 72.31\pm1.92 -> 56.88\pm 4.20 | Hurt |
| Remove the UNet learned prior (ablation) | LPT · Connect Four | win score: 0.99\pm0.01 -> 0.90\pm0.06 | Hurt |
| Reduce the capacity or expressiveness of the UNet prior (ablation) | LPT · walker2d-medium-replay | normalized score: 72.31 \pm 1.92 -> 64.06 \pm 1.94; 64.59 \pm 1.54; 70.49 \pm 2.84; 67.95 \pm 4.64 | Hurt |
| Use an original expressive UNet prior rather than a standard normal prior (ablation) | LPT · walker2d-medium-replay | normalized score: 56.88 \pm 4.20 -> 72.31 \pm 1.92 | Helped |
Surprising findings
- LPT underperforms on Antmaze-large, which the authors attribute to the dataset being dominated by length-1 trajectories (75.86%), a bias harmful to sequence models but not to TD methods.
- Removing the UNet prior reduces LPT performance across tasks, e.g., walker2d-medium-replay from 72.31 to 56.88.
Limitations
- Performance falls behind on Antmaze-large due to biased trajectory length distribution in the dataset.
- Relies on a Langevin MCMC posterior sampling loop at training and test time, which is costly for long horizons.
Experimental setup
Datasets: D4RL Gym-Mujoco (halfcheetah, hopper, walker2d) D4RL Antmaze umaze and umaze-diverse D4RL Maze2D umaze, medium, large Franka Kitchen mixed and partial Connect Four
Benchmarks: D4RL Gym-Mujoco Franka Kitchen Maze2D Antmaze Connect Four
Metrics: normalized score mean return win/outcome score
Baselines: CQL DT QDT ESPER
Builds on: Decision Transformer Quantile Decision Transformer Langevin dynamics Conservative Q-Learning
Closest papers by meaning
- PcLast: Discovering Plannable Continuous Latent States (ICML 2024 · similarity 0.89)
- Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models (NeurIPS 2025 · similarity 0.89)
- Efficient Planning with Latent Diffusion (ICLR 2024 · similarity 0.88)
- State-Covering Trajectory Stitching for Diffusion Planners (NeurIPS 2025 · similarity 0.88)
- Reasoning with Latent Diffusion in Offline Reinforcement Learning (ICLR 2024 · similarity 0.88)
- Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.