Research map / Image generation / Fast diffusion sampling
Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Morse, a dual-model (Dash and Dot) sampling framework that accelerates pretrained diffusion models losslessly by combining jump sampling with a lightweight residual-feedback model that shares weights with the base model.
Problem
Jump sampling speeds up diffusion models but degrades sample quality because steps between visited points are skipped; the goal is lossless acceleration across sampling-step budgets without retraining the base model.
Approach
Dash is the unchanged pretrained diffusion model run with jump sampling; Dot is a much faster network built by adding light down/up-sampling blocks around the frozen Dash weights plus LoRA, trained to output a residual that lifts Dash's noise estimate to match the non-jumped next-step estimate, using trajectory information (x_ts, z_ts, t_s). The two are interleaved in time.
Key results
- Morse yields lossless speedups of 1.78x to 3.31x on average over 9 baseline diffusion models across 6 image generation tasks (per abstract).
- On MS-COCO with Stable Diffusion v1.4, Morse gives an average speedup of 2.29x in best-FID comparison.
- With LCM-SDXL, Morse achieves an average speedup of 1.43x on CLIP score over 1 to 4 sampling steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Morse | Stable Diffusion v1.4 · MS-COCO, 10 LSDs, guidance scale 7.5 | FID: 11.75 -> 9.29 | Helped |
| Add Morse | Stable Diffusion v1.4 · MS-COCO, best guidance scale, 10 LSDs | FID: 10.65 -> 8.60 | Helped |
| Add Morse instead of PFDiff | Stable Diffusion v1.4 · MS-COCO, 10 LSDs | FID: 13.06 -> 11.67 | Helped |
| Add Morse instead of AYS | DDIM · ImageNet 64×64, 10 LSDs | FID: 29.23 -> 25.76 | Helped |
| Add Morse | LCM-SDXL · MS-COCO, text-to-image | CLIP score: 30.80 -> 30.83 | Helped |
| Add trajectory information to Dot (sample, noise estimate, and time step from Dash) (ablation) | Morse · CIFAR-10, DDIM, 10 LSDs | FID: 13.56 -> 6.60 | Helped |
| Add the Dash trajectory sample as Dot input (ablation) | Morse · CIFAR-10, DDIM, 10 LSDs | FID: 13.56 -> 7.50 | Helped |
| Add the Dash trajectory time step as Dot input (ablation) | Morse · CIFAR-10, DDIM, 10 LSDs | FID: 13.56 -> 8.11 | Helped |
| Add the Dash trajectory noise estimate as Dot input (ablation) | Morse · CIFAR-10, DDIM, 10 LSDs | FID: 13.56 -> 8.06 | Helped |
| Use trainable sampling blocks instead of direct resolution transformation (ablation) | Morse Dot for Stable Diffusion · MS-COCO, 10 LSDs, best guidance scale | FID: 370.79 -> 9.23 | Helped |
| Add LoRA for Dot training (ablation) | Morse Dot for Stable Diffusion · MS-COCO, 10 LSDs, best guidance scale | FID: 370.79 -> 9.79 | Helped |
| Use trainable sampling blocks and LoRA together (ablation) | Morse Dot for Stable Diffusion · MS-COCO, 10 LSDs, best guidance scale | FID: 9.23 -> 8.60; 9.79 -> 8.60 | Helped |
Surprising findings
- Without trajectory information, the Dot model cannot accelerate diffusion models, since its estimates are inferior.
- Speedups sharply drop below 1.0x when more than about 70% of steps are exchanged to Dot.
Limitations
- Adds an extra Dot model with trainable parameters (e.g., 97.84M for Stable Diffusion), which adds deployment overhead.
- Speedup depends on the Dot-to-Dash latency ratio N and degrades when too many steps are handled by Dot.
- On Stable Diffusion, FID does not improve monotonically with step count, so some evaluation schemes needed custom selection.
Experimental setup
Datasets: CIFAR-10 CelebA ImageNet CelebA-HQ LSUN-Church LAION-5B MS-COCO
Benchmarks: CIFAR-10 ImageNet 64x64 CelebA CelebA-HQ LSUN-Church MS-COCO
Metrics: FID CLIP score speedup
Baselines: DeepCache PFDiff AYS LCM-SDXL DDIM DDPM DPM-Solver
Builds on: DDIM DPM-Solver Latent Diffusion / Stable Diffusion LoRA Latent Consistency Models
Closest papers by meaning
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.92)
- T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching (ICLR 2025 · similarity 0.90)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.90)
- Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network (NeurIPS 2026 · similarity 0.90)
- Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration (NeurIPS 2025 · similarity 0.90)
- TADA: Improved Diffusion Sampling with Training-free Augmented DynAmics (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.