Research map / Image generation / Fast diffusion sampling
Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Hyper-SD, combining trajectory segmented consistency distillation, human feedback learning, and score (DMD) distillation into unified LoRAs that give SOTA 1-8 step image generation for SD1.5 and SDXL.
Problem
Multi-step diffusion inference is expensive, and existing distillation methods either degrade quality (trajectory-preserving) or cause domain shift (trajectory-reformulating) when reducing steps.
Approach
Segment the timestep range into k pieces and perform consistency distillation within segments, progressively reducing k (8→4→2→1) to achieve all-time consistency; add human/aesthetic and perceptual (instance segmentation) feedback, and DMD-style score distillation for one-step quality, delivered as a unified LoRA.
Key results
- Hyper-SDXL 1-step achieves CLIP 32.85, Aes 5.85, ImageReward 1.19, PickScore 0.231, versus SDXL-Turbo 32.33/5.33/0.78/0.228.
- Hyper-SDXL 1-step surpasses SDXL-Lightning by +0.68 CLIP Score and +0.51 Aes Score per the abstract.
- Hyper-SD15 1-step reaches CLIP 30.87 and Aes 5.79, outperforming SD15-LCM and SD15-TCD at 4 steps on aesthetics and ImageReward.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace TCD with TSCD (ablation) | SDXL distillation without human feedback · COCO-5k, 2 steps | Image Reward: 0.29 -> 0.64 | Helped |
| replace TCD with TSCD (ablation) | SDXL distillation without human feedback · COCO-5k, 4 steps | CLIP Score: 32.45 -> 32.53 | Helped |
| replace TCD with TSCD (ablation) | SDXL distillation without human feedback · COCO-5k, 8 steps | CLIP Score: 32.47 -> 32.46 | No clear effect |
| add human feedback learning (ablation) | TSCD · COCO-5k, SDXL, 1 step | Image Reward | Helped |
| replace TCD+DMD with TSCD+DMD (ablation) | one-step SDXL distillation · COCO-5k, 1 step | Image Reward: 0.31 -> 0.85 (+0.54) | Helped |
| replace TCD+RLHF with TSCD+RLHF (ablation) | SDXL distillation with human feedback · COCO-5k, 2 steps | Image Reward: 0.59 -> 1.04 (+0.45) | Helped |
| replace TCD with TSCD (ablation) | SDXL distillation · COCO-5k, 2 steps | Aes Score: 5.62 -> 5.58 (-0.04) | Hurt |
| use one unified LoRA across inference step counts (ablation) | Hyper-SD · COCO-5k, SDXL, 1, 2, 4, and 8 steps | CLIP Score: 32.54; 32.51; 32.59; 32.59 | No clear effect |
Surprising findings
- Human feedback learning lets the distilled models score higher than the original SDXL base on aesthetics and ImageReward at fewer steps.
- The unified LoRA gives nearly identical CLIP Score across 1-8 steps.
Limitations
- Eliminates classifier-free guidance, so negative prompts cannot be used.
- Relies on generic reward models rather than customized feedback.
- Uses UNet architectures; diffusion transformer architectures not explored.
Experimental setup
Datasets: LAION COYO COCO2017 COCO-5k
Benchmarks: COCO-5k
Metrics: CLIP Score Aes Score ImageReward PickScore user study
Baselines: LCM TCD PeRFlow SDXL-Turbo SDXL-Lightning DMD SD21-Turbo
Builds on: Consistency Distillation Consistency Trajectory Model Distribution Matching Distillation ReFL SDXL-Lightning
Closest papers by meaning
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.89)
- Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps (NeurIPS 2024 · similarity 0.89)
- Continuous-Time Distribution Matching for Few-Step Diffusion Distillation (NeurIPS 2026 · similarity 0.88)
- SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent Trajectories (NeurIPS 2025 · similarity 0.87)
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.87)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.