Research map / Image generation / Fast diffusion sampling
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T-Stitch, a training-free method that runs a small diffusion model for early denoising steps and switches to a large one for later steps, speeding up sampling with little or no quality loss.
Problem
Sampling from large diffusion probabilistic models for high-quality image generation is expensive because it requires many denoising steps with a large model at each step.
Approach
Use a small pretrained DPM for the early (low-frequency, global-structure) denoising steps and a large DPM for later steps; pretrained models of different sizes trained on the same data share similar latent trajectories, so they can be stitched along the sampling path without retraining.
Key results
- On DiT-XL, replacing the first 40% of DDIM steps with DiT-S causes no FID degradation (about 1.5x lossless speedup on ImageNet).
- Three-model T-Stitch (DiT-S/B/XL) at ~10s reaches FID 9.21 vs 9.19 for DiT-XL alone with 1.7x speedup.
- With BK-SDM Tiny stitched into SD v1.4, the early 30% of steps yields FID 12.54 vs 13.07 baseline with CLIP score nearly unchanged, and time cost drops from 3.1s to 2.8s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiT-S for the first 40% of steps instead of DiT-XL throughout | DiT-XL sampling · ImageNet-256 | FID: 9.20 -> 8.95 | No clear effect |
| Use DiT-S for the first 50% of steps instead of DiT-XL throughout | DiT-XL sampling · ImageNet-256 | speedup: 1.76× | Helped |
| Use LDM-S for the first 40% of steps | LDM sampling · ImageNet-256 | FID: 20.11 -> 18.60 | Helped |
| Use BK-SDM Tiny for the first 20% of steps | SD v1.4 sampling · MS-COCO 256×256 | FID: 13.07 -> 12.29 | Helped |
| Use BK-SDM Tiny for the first 20% of steps | SD v1.4 sampling · MS-COCO 256×256 | CLIP score: 0.2957 -> 0.2938 | No clear effect |
| Use DiT-S, then DiT-B, then DiT-XL (50%:30%:20%) (ablation) | DiT-XL sampling · ImageNet-256 | FID: 16.49 -> 13.35 | Helped |
| Finetune at the stitched intervals rather than all timesteps (ablation) | T-Stitch DiT-S/DiT-B/DiT-XL · ImageNet-256 | FID: 16.04 -> 13.35 | Helped |
| Use DiT-S first, then DiT-XL, rather than interleaving the models (ablation) | DiT-S/DiT-XL sampling · ImageNet-256 | FID: 19.02 -> 10.06 | Helped |
| Use DiT-S first, then DiT-XL, rather than decreasing the small-model probability (ablation) | DiT-S/DiT-XL sampling · ImageNet-256 | FID: 12.94 -> 10.06 | Helped |
| Use DiT-S first, then DiT-XL, rather than using DiT-XL first (ablation) | DiT-S/DiT-XL sampling · ImageNet-256 | FID: 27.61 -> 10.06 | Helped |
| Increase the DiT-S fraction from 40% to 50% (ablation) | T-Stitch DiT-S/DiT-XL · ImageNet-256 | FID: 8.95 -> 10.06 | Hurt |
| Increase the LDM-S fraction from 40% to 50% (ablation) | T-Stitch LDM-S/LDM · ImageNet-256 | FID: 18.60 -> 19.33 | Hurt |
Surprising findings
- Stitching a small Stable Diffusion model improved FID and prompt alignment (CLIP score) for stylized SD models such as InkPunk Diffusion.
- Small-to-large ordering works much better than large-to-small (FID 10.06 vs 27.61 in the stitching-order ablation).
Limitations
- Requires a smaller model trained on the same data distribution as the large model, so speedup is bounded by the small model's efficiency.
- Adds a small model's memory and storage overhead, though minor relative to the large model.
- Non-negligible performance drop when progressively using a small model, which needs finetuning at the allocated intervals to mitigate.
Experimental setup
Datasets: ImageNet-1K MS-COCO
Benchmarks: ImageNet 256x256 class-conditional generation MS-COCO 256x256
Metrics: FID Inception Score CLIP score Precision Recall time cost
Baselines: SN-Netv2 model stitching (M-Stitch) reducing sampling steps DeepCache ToMe LCM
Builds on: DiT DDIM classifier-free guidance stitchable neural networks speculative decoding
Closest papers by meaning
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.90)
- Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models (ICML 2025 · similarity 0.90)
- PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores (ICLR 2025 · similarity 0.90)
- Stitched Value Model for Diffusion Alignment (NeurIPS 2026 · similarity 0.90)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.90)
- Accelerating Diffusion Sampling with Optimized Time Steps (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.