Research map / Label-efficient and robust learning / Knowledge distillation
LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models
Hyunsoo Han, Sangyeop Yeo, Jaejun Yoo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive coarse-to-fine weight instead of linear schedule (ablation) | LIFT · CelebA 90% pruned | FID: 18.09 -> 15.73 | Improved |
| add adaptive coarse-to-fine weight instead of cosine schedule (ablation) | LIFT · CelebA 90% pruned | FID: 17.45 -> 15.73 | Improved |
| add PLACE to LIFT with FeatKD (ablation) | LIFT · CelebA 90% pruned, 78.7M teacher | FID: 18.49 -> 15.73 | Improved |
| add PLACE to LIFT without FeatKD (ablation) | LIFT · CelebA 90% pruned, 78.7M teacher | FID: 19.07 -> 16.67 | Improved |
| add FeatKD to LIFT+PLACE (ablation) | LIFT+PLACE · CelebA 90% pruned, 78.7M teacher | FID: 16.67 -> 15.73 | Improved |
| replace OutKD+FeatKD with LIFT/PLACE at 90% pruning | KD · CelebA 90% pruned | FID: 211.23 -> 15.73 | Improved |
| use OutKD instead of fine-tuning without KD at 50% and 70% pruning | OutKD · CelebA | FID: 5.30 -> 5.53 and 6.32 -> 6.46 | Hurt |
| replace OutKD+FeatKD with LIFT/PLACE at 30% pruning | KD · LSUN-Bedroom | FID: 23.35 -> 16.57 | Improved |
| use strongest 78.7M teacher with conventional OutKD+FeatKD instead of 9.2M teacher (ablation) | OutKD+FeatKD · CelebA 1.3M student | FID: 40.88 -> 193.56 | Hurt |
| use strongest 78.7M teacher with LIFT/PLACE (ablation) | LIFT+PLACE · CelebA 1.3M student | FID: 25.38 -> 17.03 | Improved |
| use too small group size (K=2^3) or too large (K=2^6) in PLACE (ablation) | PLACE · CelebA 90% pruned | FID, precision, recall | Hurt |
| replace OutKD with LIFT/PLACE for BK-Base-v2 | BK-SDM · LAION SD 2.1 | FID: 15.85 -> 16.72 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- FeatKD — FEAT-KD: Learning Concise Representations for Single and Multi-Target Regression via TabNet Knowledge Distillation (ICML 2025) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.90)
- DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture (CVPR 2025 · similarity 0.89)
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.89)
- Data-free Distillation of Diffusion Models with Bootstrapping (ICML 2024 · similarity 0.89)
- Progressively Knowledge Distillation via Re-parameterizing Diffusion Reverse Process (AAAI 2024 · similarity 0.89)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
