Research map / Image generation / Fast diffusion sampling
Early-Bird Diffusion: Investigating and Leveraging Timestep-Aware Early-Bird Tickets in Diffusion Models for Efficient Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EB-Diff-Train, which finds early-bird sparse subnetworks (including timestep-region-specific pruning rates) in diffusion models, trains them in parallel, and ensembles them at inference to cut DM training cost.
Problem
Training diffusion models requires heavy compute because of many forward/backward passes over many timesteps, and standard train-prune-finetune pipelines still need a fully trained dense model first.
Approach
Extends Early-Bird lottery tickets to diffusion models, and introduces timestep-aware EB tickets with different sparsity per timestep region (acceleration, deceleration, convergence), trained in parallel and combined at inference.
Key results
- On CIFAR-10 with DDPM, EB-Diff-Train (TA-EB) reaches up to 10.32x speedup over train-prune-finetune baselines with comparable FID.
- On CelebA, EB-Diff-Train (TA-EB) achieves 4.81x to 6.74x speedups with comparable or lower FID than baselines.
- On LSUN Church, EB-Diff-Train (EB) achieves 2.10x speedup with FID 37.60 versus 54.18 for scratch training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a 2% overlap at timestep-region boundaries (ablation) | TA-EB · CIFAR-10 | FID: 8.10 -> 7.69 | Helped |
| increase the third-region pruning rate from 80.0% to 90.0% (30.0%/40.0%/90.0%) (ablation) | TA-EB · CIFAR-10 | FID: 7.48 -> 17.61 | Hurt |
| use 30.0%/60.0%/80.0% pruning across timestep regions instead of 30.0%/40.0%/80.0% (ablation) | TA-EB · CIFAR-10 | FID: 7.75; 7.69 | No clear effect |
| vary the training iterations allocated to each timestep region (ablation) | TA-EB · CIFAR-10 | FID: 7.69; 7.56; 7.64; 7.40; 7.49; 7.60; 7.49 | Mixed |
| combine EB-Diff-Train with SpeeD timestep resampling (ablation) | EB-Diff-Train · CIFAR-10 32×32 | FID: 9.32 -> 8.89; 9.34 -> 8.01 | Helped |
| use an early-bird ticket rather than a scratch-pruned model | DDPM · LSUN Church 256×256 | FID: 54.18 -> 37.60 | Helped |
| use an early-bird ticket rather than a scratch-pruned model | DDPM · LSUN Bedroom 256×256 | FID: 217.07 -> 81.60 | Helped |
| use an early-bird ticket rather than a scratch-pruned model | LDM · ImageNet-1K 256×256 | FID: 43.10 -> 36.18 | Helped |
| use timestep-aware tickets rather than an early-bird ticket | DDPM · CIFAR-10 32×32 | speedup: 1.28× -> 5.78× | Helped |
| use timestep-aware tickets rather than an early-bird ticket | DDPM · CelebA 64×64 | speedup: 1.89× -> 4.38× | Helped |
Surprising findings
- Magnitude pruning consistently identifies the earliest EB tickets, outperforming the more costly Taylor and Diff-Pruning criteria in finding time.
Limitations
- Experiments use only DDPM and LDM on five datasets, with pruning hyperparameters borrowed from Diff-Pruning.
- Timestep region partitioning and per-region pruning rates are hand-set from prior work rather than learned.
- Some TA-EB configurations with aggressive pruning show notable FID degradation (e.g., 17.61 for one 65.6% average-pruning setting).
Experimental setup
Datasets: CIFAR-10 CelebA LSUN Bedroom LSUN Church ImageNet-1K
Benchmarks: CIFAR-10 CelebA LSUN Church LSUN Bedroom ImageNet-1K
Metrics: FID #Params MACs training speed-up
Baselines: Scratch Random pruning Magnitude pruning Taylor pruning Diff-Pruning SpeeD
Builds on: Early-Bird tickets Diff-Pruning Lottery Ticket Hypothesis SpeeD
Closest papers by meaning
- Adaptive Non-Uniform Timestep Sampling for Accelerating Diffusion Model Training (CVPR 2025 · similarity 0.90)
- Improving Training Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architecture (CVPR 2024 · similarity 0.90)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.89)
- Analyzing and Improving the Training Dynamics of Diffusion Models (CVPR 2024 · similarity 0.89)
- On improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.