Research map / Image generation / Image restoration
Random Conditioning with Distillation for Data-Efficient Diffusion Model Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes random conditioning, pairing noised images with randomly sampled text conditions during distillation, enabling image-free compression of Stable Diffusion into smaller student UNets that generate concepts unseen in the training images.
Problem
Knowledge distillation of conditional diffusion models requires generating paired images for all text conditions, which is costly, and students fail to generate concepts absent from the distillation images.
Approach
Pair each noised training image with a random text condition from a larger text pool with timestep-dependent probability p(t), so the student learns the teacher's behavior on conditions for which no images were generated, without extra image generation.
Key results
- Random conditioning lowers FID from 15.76 to 15.00 and raises IS from 36.03 to 36.14 over the teacher-initialized baseline; without teacher init it lowers FID 18.13 to 15.46 (Table 1).
- B-Base with random conditioning and no real images reaches FID 14.47, IS 36.50, CLIP 0.2932, versus BK-SDM-Base at FID 15.76 and teacher FID 13.05.
- Trained without animal images, random conditioning with 24K animal texts plus 20M LAION texts improves unseen-animal FID from 37.86 to 24.71 (Table 2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add random conditioning (ablation) | B-Base image-free distillation with random initialization · MS-COCO-30k | FID: 18.13 -> 15.46 | Helped |
| Add random conditioning (ablation) | B-Base image-free distillation with random initialization · MS-COCO-30k | IS: 31.84 -> 34.48 | Helped |
| Add teacher initialization (ablation) | B-Base without random conditioning or real images · MS-COCO-30k | IS: 31.84 -> 33.81 | Helped |
| Add real images to teacher-initialized training without random conditioning (ablation) | B-Base · MS-COCO-30k | FID: 18.15 -> 15.76 | Helped |
| Add real images to random-conditioned, teacher-initialized training (ablation) | B-Base · MS-COCO-30k | IS: 36.03 -> 36.14 | No clear effect |
| Add random conditioning with the 24K animal-related texts excluded from image training (ablation) | B-Base trained on non-animal images · Unseen (Animal) | FID: 37.86 -> 23.26 | Helped |
| Add random conditioning with the 24K animal-related texts excluded from image training (ablation) | B-Base trained on non-animal images · Unseen (Animal) | CLIP score: 0.2478 -> 0.2833 | Helped |
| Add 20M LAION texts to random conditioning (ablation) | B-Base trained on non-animal images · Unseen (Animal) | CLIP score: 0.2833 -> 0.2913 | Helped |
| Add random conditioning using LAION text (ablation) | B-Base image-free distillation · MS-COCO 30K | FID: 18.15 -> 15.76 | Helped |
| Replace LAION text with GPT-generated prompts for random conditioning (ablation) | B-Base image-free distillation · MS-COCO 30K | FID: 15.76 -> 14.98 | Helped |
| Use exponentially varying random-conditioning probability instead of a constant probability (ablation) | Random conditioning · Text-to-image diffusion distillation | generation performance | Helped |
| Reduce generated-image cache size while using random conditioning (ablation) | B-Base · MS-COCO 30K | FID | Helped |
Surprising findings
- Random conditioning with 10K generated images outperforms the model trained on 212K images without it.
- C-Micro, with about 50% fewer UNet parameters than BK-SDM-Small, surpasses it on all metrics while using no real images and random initialization.
Limitations
- Experiments limited to text-to-image Stable Diffusion v1.4 teacher; not validated on other modalities or newer teachers like SDXL.
- Probability schedule p(t) chosen empirically; constant p(t) gave suboptimal results.
- Evaluation relies on FID, IS, and CLIP score on MS-COCO 30K without human evaluation.
Experimental setup
Datasets: LAION-Aesthetics V2 6.5+ (212K) LAION 400M text prompts MS-COCO 30K validation MNIST
Benchmarks: MS-COCO 30K
Metrics: FID IS CLIP score
Baselines: BK-SDM-Base BK-SDM-Small BK-SDM-Tiny SDM-v1.4 Small SD KOALA
Builds on: BK-SDM Stable Diffusion feature distillation
Closest papers by meaning
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.89)
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.86)
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.85)
- Data-free Distillation of Diffusion Models with Bootstrapping (ICML 2024 · similarity 0.85)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.85)
- Simple Distillation for One-Step Diffusion Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.