Research map / Image generation / Fast diffusion sampling
A First-order Generative Bilevel Optimization Framework for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a first-order generative bilevel framework for diffusion models that tunes fine-tuning entropy strength via inference-only guided sampling and noise schedules via zeroth-order estimation during training.
Problem
Optimizing diffusion models for downstream objectives involves nested bilevel structures (e.g., reward fine-tuning entropy strength, noise schedules), but traditional bilevel methods fail in infinite-dimensional distribution spaces and with costly sampling.
Approach
Treats the lower-level distribution via guided backward sampling (inference-only) and uses a penalty-based first-order hypergradient; for scratch training, reparameterizes the lower-level as score-network parameters and estimates the upper-level gradient for a noise scheduler with zeroth-order methods and a closed-form score-matching gradient.
Key results
- Reward fine-tuning with ImageNet ResNet-18 synthetic reward: bilevel method FID 102.82 and CLIP 39.54 vs best baseline weighted sum FID 116.52 and CLIP 36.50 at similar time (3.35h vs 4.31h).
- Reward fine-tuning with HPSv2 reward: bilevel FID 137.23 and CLIP 37.10, better than weighted sum (140.56, 35.40).
- Noise scheduling on MNIST: bilevel cosine FID 65.41 and IS 1.78 in 3.88h, comparable to grid search (67.30 FID, 31.53h) and about 6x faster.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use bilevel tuning instead of the weighted-sum baseline | StableDiffusion V1.5 fine-tuning with synthetic lower-level reward · Fine-tuning, synthetic lower-level reward | FID: 116.52 \ {\scriptstyle \pm 3.9} -> 102.82 \ {\scriptstyle \pm 3.5} | Helped |
| Use bilevel tuning instead of the weighted-sum baseline | StableDiffusion V1.5 fine-tuning with synthetic lower-level reward · Fine-tuning, synthetic lower-level reward | CLIP: 36.50 \ {\scriptstyle \pm 1.1} -> 39.54 \ {\scriptstyle \pm 2.2} | Helped |
| Use bilevel tuning instead of the weighted-sum baseline | StableDiffusion V1.5 fine-tuning with synthetic lower-level reward · Fine-tuning, synthetic lower-level reward | Time (hours): 4.31 -> 3.35 | Helped |
| Use bilevel tuning instead of the weighted-sum baseline | StableDiffusion V1.5 fine-tuning with HPSv2 reward · Fine-tuning, HPSv2 lower-level reward | FID: 140.56 \ {\scriptstyle \pm 1.3} -> 137.23 \ {\scriptstyle \pm 1.7} | Helped |
| Use bilevel tuning instead of grid search | StableDiffusion V1.5 fine-tuning with HPSv2 reward · Fine-tuning, HPSv2 lower-level reward | CLIP: 36.3 \ {\scriptstyle \pm 1.7} -> 37.10 \ {\scriptstyle \pm 3.2} | Helped |
| Optimize the schedule bilevelly instead of Bayesian search | DDIM noise scheduling on MNIST with cosine parameterization · MNIST, cosine schedule | FID: 67.16 -> 65.41 | Helped |
| Optimize the schedule bilevelly instead of Bayesian search | DDIM noise scheduling on MNIST with cosine parameterization · MNIST, cosine schedule | IS: 1.69 -> 1.78 | Helped |
| Optimize the schedule bilevelly instead of Bayesian search | DDIM noise scheduling on MNIST with cosine parameterization · MNIST, cosine schedule | Time (hours): 26.85 -> 3.88 | Helped |
| Optimize the schedule bilevelly instead of Bayesian search | DDIM noise scheduling on MNIST with sigmoid parameterization · MNIST, sigmoid schedule | FID: 65.17 -> 65.16 | Helped |
| Optimize the schedule bilevelly instead of Bayesian search | DDIM noise scheduling on MNIST with sigmoid parameterization · MNIST, sigmoid schedule | IS: 1.65 -> 1.79 | Helped |
| Optimize the schedule bilevelly instead of Bayesian search | DDIM noise scheduling on MNIST with sigmoid parameterization · MNIST, sigmoid schedule | Time (hours): 29.13 -> 3.94 | Helped |
| Learn the noise schedule bilevelly instead of using default DDIM parameters | DDIM noise scheduling on MNIST with cosine parameterization · MNIST, cosine schedule | FID: 105.27 -> 65.41 | Helped |
Surprising findings
- On MNIST noise scheduling, the bilevel method is comparable to grid and Bayesian search rather than clearly superior, and Bayesian search yields images focused on simple digits such as 1 and 7.
Limitations
- Theoretical guarantees rely on strong convexity and smoothness assumptions that are argued rather than fully verified for diffusion models.
- Noise-scheduling experiments are limited to MNIST with a small U-Net.
- Fine-tuning relies on a differentiable CLIP upper-level reward and a synthetic or HPSv2 lower-level reward.
Experimental setup
Datasets: ImageNet MNIST
Benchmarks: MNIST HPSv2
Metrics: FID CLIP score Inception Score running time
Baselines: Grid search Random search Bayesian search Weighted sum DDIM (default) Greedy grid search
Builds on: equilibrium backpropagation guided diffusion for generative optimization DDPM DDIM
Closest papers by meaning
- Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models (CVPR 2025 · similarity 0.90)
- CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation (NeurIPS 2025 · similarity 0.89)
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models (ICML 2024 · similarity 0.88)
- Diffusion Models are Evolutionary Algorithms (ICLR 2025 · similarity 0.88)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.88)
- Denoising Task Difficulty-based Curriculum for Training Diffusion Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.