Research map / Image generation / Fast diffusion sampling
Efficient Hybrid Distillation: Synergizing Score and Adversarial Objectives for One-Step Diffusion
Fei Peng, Junqiang Wu, Haoxian Tan, Jun Zhou, Jie Hu, Xiaoming Wei, Jie Guo, Xiu Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hybrid distillation approach that synergizes score-based and adversarial objectives to produce one-step diffusion generators.
Problem
Accelerating diffusion models via distillation into one-step generators; existing hybrid methods combine distillation objectives simply, leaving effective synergy underexplored.
Approach
Focuses on how to effectively synergize score distillation and adversarial objectives rather than simply summing them, for one-step diffusion distillation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Regime-Adaptive Sampling (score high-noise, adversarial low-noise) instead of uniform (ablation) | EASD, SiT-XL/2+REPA · ImageNet 256x256 | FID: 1.60 -> 1.47 | Improved |
| hard truncation instead of uniform timestep sampling (ablation) | EASD · ImageNet 256x256 | FID: 1.60 -> 1.51 | Improved |
| soft Logit-Normal biasing instead of hard truncation (ablation) | EASD · ImageNet 256x256 | FID: 1.51 -> 1.47 | Improved |
| add Regime-Adaptive Sampling to DMD2 (ablation) | DMD2 · ImageNet 64x64 | FID@100K / FID@200K: 1.72 -> 1.50; 1.52 -> 1.35 | Improved |
| replace fake-score-updating adversarial step with frozen-teacher discriminator (ablation) | DMD2 with EASD · ImageNet 64x64 | FID@100K: 1.50 -> 1.46; 28h -> 23h | Improved |
| bias score distillation to low noise (ablation) | Score distillation only · ImageNet 256x256 | FID: 2.14 -> 2.99 | Hurt |
| bias adversarial distillation to high noise (ablation) | Adversarial distillation only · ImageNet 256x256 | FID: 2.31 -> 2.80 | Hurt |
| attach discriminator to frozen teacher instead of updating fake score (ablation) | EASD · ImageNet 256x256 | FID: 1.85 -> 1.47 | Improved |
| attach discriminator at middle block instead of shallow or deep (ablation) | EASD · ImageNet 256x256 | FID: 1.55 (block 7), 1.70 (block 21) -> 1.47 | Improved |
| add Fake Score EMA to uniform-sampling baseline (ablation) | EASD baseline · ImageNet 256x256 | FID: 1.60 -> 1.56 | Improved |
| add Iterative Warmup (ablation) | EASD · ImageNet 256x256 | FID: 1.41 -> 1.39 (60 epochs) | Improved |
| add Latent Mode Seeking loss (ablation) | EASD baseline · ImageNet 256x256 | FID: 1.60 -> 1.58 | Improved |
Closest papers by meaning
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.90)
- Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch (NeurIPS 2025 · similarity 0.89)
- Adversarial Score Distillation: When score distillation meets GAN (CVPR 2024 · similarity 0.89)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.89)
- Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation (ICLR 2025 · similarity 0.89)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
