Research map / Image generation / Fast diffusion sampling
Scale-wise Distillation of Diffusion Models
Nikita Starodubcev, Ilya Drobyshevskiy, Denis Kuznedelev, Artem Babenko, Dmitry Baranchuk
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| upsample x0 prediction then renoise (B) instead of noise then upscale (C) (ablation) | SD3.5 noisy latents · COCO, 64->128, t=400 | FID-5K: 122 -> 30.9 | Improved |
| scale-wise sampling instead of full-resolution at 4 steps (ablation) | SD3.5-M distilled · COCO 30K | HPSv3: 11.3 -> 11.7 | Improved |
| scale-wise instead of full-resolution at 6 steps (ablation) | SD3.5-M distilled · COCO 30K | FID: 20.4 -> 23.1 | Hurt |
| scale-wise instead of full-resolution at 2 steps (ablation) | SD3.5-M distilled · COCO 30K | HPSv3: 10.8 -> 10.6 | Improved |
| scale-wise 4 steps vs full-scale 2 steps at matched time (ablation) | SD3.5-M · COCO 30K | HPSv3: 10.8 -> 11.7 | Improved |
| scale-wise vs full-scale sampling time (4 steps) (ablation) | SD3.5-M · sampling latency | sec/image: 0.29 -> 0.17 | Improved |
| scale-wise vs full-scale training iteration (full objective) (ablation) | Wan2.1 · training time | sec/iteration: 70.6 -> 23.9 | Improved |
| add temporal scaling to spatial-only SwD (ablation) | SwD on Wan2.1-1.3B · MovieGenBench | latency s/video: 2.1 -> 1.8 | Improved |
| add temporal scaling to spatial-only SwD (ablation) | SwD on Wan2.1-1.3B · MovieGenBench | VBench2 Overall: 52.8 -> 53.2 | Improved |
| use MMD loss alone instead of full objective (ablation) | SwD · training time | sec/iteration (SD3.5-M scale-wise): 3.2 -> 0.4 | Improved |
| remove MMD loss from L_SwD (ablation) | SwD SD3.5-M · MJHQ | automatic metrics | Improved |
| average feature tokens across batch (B) or extract features from clean samples (C) (ablation) | L_MMD · SD3.5-M MJHQ | automatic metrics | Hurt |
Closest papers by meaning
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.92)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.92)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.91)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.91)
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.90)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
