Research map / Image generation / Fast diffusion sampling
Noise Conditional Variational Score Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills pretrained EDM2 diffusion models into noise-conditional one-step generative denoisers via a VSD-style objective plus adversarial loss, enabling multi-step sampling and plug-and-play posterior sampling (PnP-GD).
Problem
Diffusion models are slow to sample, and few-step distilled generators lose iterative refinement and asymptotically exact zero-shot posterior sampling for inverse problems.
Approach
Shows the unconditional score implicitly gives the score of denoising posterior distributions (effective noise level), and extends VSD to condition on noisy data so a one-step conditional generator approximates the denoising posterior across noise levels; used with DDIM-like multi-step sampling and Split Gibbs Sampler inference.
Key results
- On ImageNet-512x512, 4-step NCVSD-L reaches FID 1.76, beating 2-step sCD-XXL (1.88) and EDM2-XXL (1.81).
- On ImageNet-64x64, 4-step NCVSD-L reaches FID 1.53, better than 2-step sCD-XL (1.66).
- PnP-GD uses 50 NFE on FFHQ inverse problems, achieving best or second-best LPIPS across tasks versus DAPS at 1000 NFE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase sampling from 1-step to 4-step | NCVSD-L · ImageNet-64×64 class-conditional | FID: 2.96 -> 1.53 | Helped |
| Increase sampling from 1-step to 4-step | NCVSD-L · ImageNet-512×512 class-conditional | FID: 2.56 -> 1.76 | Helped |
| Scale model size from S to L | NCVSD, 4-step · ImageNet-64×64 class-conditional | FID: 2.14 -> 1.53 | Helped |
| Scale model size from S to L | NCVSD, 4-step · ImageNet-512×512 class-conditional | FID: 2.00 -> 1.76 | Helped |
| Use 4-step sampling instead of 2-step sampling | NCVSD-L · ImageNet-64×64 class-conditional | FID: 2.35 -> 1.53 | Helped |
| Use 4-step sampling instead of 2-step sampling | NCVSD-L · ImageNet-512×512 class-conditional | FID: 2.03 -> 1.76 | Helped |
| Use NCVSD-L with 4-step sampling instead of EDM2-XXL | ImageNet-512×512 class-conditional generation · ImageNet-512×512 class-conditional | FID: 1.81 -> 1.76 | Helped |
| Use NCVSD-L with 4-step sampling instead of sCD-XXL with 2-step sampling | ImageNet-512×512 class-conditional generation · ImageNet-512×512 class-conditional | FID: 1.88 -> 1.76 | Helped |
| Use NCVSD-L with 4-step sampling instead of sCD-XL with 2-step sampling | ImageNet-64×64 class-conditional generation · ImageNet-64×64 class-conditional | FID: 1.66 -> 1.53 | Helped |
| Use PnP-GD instead of DAPS | Noisy inverse problem solving · FFHQ-256×256 inpainting (box), noisy | LPIPS: 0.133 -> 0.128 | Helped |
| Use PnP-GD instead of DAPS | Noisy inverse problem solving · FFHQ-256×256 Gaussian deblurring, noisy | LPIPS: 0.165 -> 0.155 | Helped |
| Use PnP-GD instead of DAPS | Noisy inverse problem solving · FFHQ-256×256 motion deblurring, noisy | LPIPS: 0.157 -> 0.160 | Hurt |
Surprising findings
- Directly using the noise-conditional generator without added stochasticity led to severe mode collapse, requiring injected noise into the conditioning input.
Limitations
- Relies on pretrained diffusion teachers, so training a denoiser from scratch is not possible.
- State-of-the-art results require adversarial training that needs careful tuning for stable convergence.
- Not validated beyond the current model scale due to computational constraints.
Experimental setup
Datasets: ImageNet 64x64 ImageNet 512x512 FFHQ 256x256
Benchmarks: ImageNet 64x64 ImageNet 512x512 FFHQ 256x256 noisy inverse problems
Metrics: FID NFE LPIPS PSNR
Baselines: EDM2 consistency distillation (CD) ECM sCD DDRM DPS DiffPIR PiGDM DWT-Var DAPS PnP-DM
Builds on: Variational Score Distillation EDM2 Split Gibbs Sampler PnP-DM Consistency Models
Closest papers by meaning
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.90)
- Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models (AAAI 2024 · similarity 0.90)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.90)
- Simple Distillation for One-Step Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Score Distillation via Reparametrized DDIM (NeurIPS 2024 · similarity 0.90)
- EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.