Research map / Reinforcement learning / Visual and representation RL
State-Covering Trajectory Stitching for Diffusion Planners
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCoTS, a reward-free trajectory augmentation method that stitches short offline segments guided by a temporal-distance-preserving latent space, directional progress and novelty, plus diffusion-based refinement, to boost diffusion planners and offline GCRL on OGBench.
Problem
Diffusion planners are limited by the quality, diversity and coverage of offline training data, which restricts generalization beyond the training distribution and to longer planning horizons.
Approach
Iteratively stitch short trajectory segments into longer, diverse trajectories without reward, selecting segments by a learned latent temporal-distance embedding (nearest neighbors), a progress score along a random latent direction, and a particle-based novelty score, then smooth stitch points with a diffusion stitcher.
Key results
- SCoTS-augmented data raises HIQL's average success over OGBench Stitch/Explore tasks from 36.4 to 73.4 and CRL from 8.4 to 27.9.
- HD planner trained on SCoTS data reaches 100% on PointMaze-Giant-Stitch and 87% on AntMaze-Giant-Stitch with replanning, outperforming Compositional Diffuser (68% and 65%).
- On D4RL, Diffuser trained with SCoTS data improves average normalized return from 79.5 to 87.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train GCIQL on SCoTS-augmented data instead of original data | GCIQL · OGBench GCRL average | success rate: 12.6 -> 20.7 | Helped |
| Train CRL on SCoTS-augmented data instead of original data | CRL · OGBench GCRL average | success rate: 8.4 -> 27.9 | Helped |
| Train HIQL on SCoTS-augmented data instead of original data | HIQL · OGBench GCRL average | success rate: 36.4 -> 73.4 | Helped |
| Guide stitching with learned temporal distance-preserving embeddings rather than raw state-space distances (ablation) | SCoTS-trained HD · AntMaze Stitch | success rate: 45 ± 1 -> 93 ± 1; 7 ± 2 -> 87 ± 2 | Helped |
| Add diffusion-based stitching refinement (ablation) | SCoTS data for HD and HIQL · AntMaze Stitch | success rate: HD: 85 ± 3 -> 93 ± 1; 53 ± 1 -> 87 ± 2; HIQL: 52 ± 2 -> 91 ± 2; 11 ± 2 -> 55 ± 5 | Helped |
| Add novelty score with β=2 rather than use directional progress alone (β=0) (ablation) | SCoTS-trained HD · AntMaze Stitch | success rate: 87 ± 2 -> 93 ± 1; 63 ± 3 -> 87 ± 2 | Helped |
| Increase novelty weight from β=2 to β=4 (ablation) | SCoTS-trained HD · AntMaze Stitch | success rate: 93 ± 1 -> 92 ± 1; 87 ± 2 -> 89 ± 2 | Mixed |
| Set novelty weight to β=8 instead of β=2 (ablation) | SCoTS-trained HD · AntMaze Stitch | success rate: 93 ± 1 -> 85 ± 3; 87 ± 2 -> 74 ± 3 | Hurt |
| Use shorter sub-trajectories H=26 rather than H=104 (ablation) | SCoTS · AntMaze Explore Large | success rate: 88 ± 2 -> 98 ± 1 | Helped |
| Vary sub-trajectory length from H=26 to H=52 (ablation) | SCoTS · AntMaze Stitch Giant | success rate: 87 ± 2 -> 89 ± 2 | No clear effect |
| Retrieve K=10 rather than K=3 candidate segments (ablation) | SCoTS · AntMaze Giant Stitch | success rate: 65 ± 3 -> 87 ± 2 | Helped |
| Increase retrieved candidates from K=10 to K=20 (ablation) | SCoTS · AntMaze Giant Stitch | success rate: 87 ± 2 -> 89 ± 2 | No clear effect |
Surprising findings
- Raw state-space distance guidance for stitching performs poorly on AntMaze-Giant (7% vs 87% with learned latent), indicating Euclidean distance in raw state space does not reflect reachability.
Limitations
- Computational overhead from training the diffusion stitcher and the augmentation process.
- Temporal distance embedding is symmetric and cannot capture asymmetric temporal distances, limiting use in irreversible or disconnected MDPs.
- Evaluated mainly on OGBench locomotion navigation tasks; dense-reward and robot experiments are limited in scope.
Experimental setup
Datasets: OGBench PointMaze Stitch OGBench AntMaze Stitch OGBench AntMaze Explore D4RL hopper-medium-v2 D4RL walker2d-medium-v2 D4RL hopper-medium-replay-v2 D4RL walker2d-medium-replay-v2
Benchmarks: OGBench PointMaze OGBench AntMaze D4RL MuJoCo
Metrics: success rate normalized average return dynamic MSE
Baselines: GCIQL QRL CRL HIQL Generative Skill Chaining Compositional Diffuser SynthER DiffStitch
Builds on: Hierarchical Diffuser Diffuser implicit Q-learning Quasimetric RL DiT1D
Closest papers by meaning
- Generative Trajectory Stitching through Diffusion Composition (NeurIPS 2025 · similarity 0.93)
- DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching (ICML 2024 · similarity 0.91)
- Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning (ICML 2025 · similarity 0.89)
- Simple Hierarchical Planning with Diffusion (ICLR 2024 · similarity 0.89)
- Stitching Sub-trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL (AAAI 2024 · similarity 0.89)
- Structural Information-based Hierarchical Diffusion for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.