Research map / Image generation / Fast diffusion sampling
Distributional Diffusion Models with Scoring Rules
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Distributional Diffusion Models that replace the MSE denoiser regression with a generalized energy/kernel scoring rule so a generator can sample the posterior p(x0|xt), enabling accurate few-step sampling.
Problem
Diffusion models need many denoising steps because the Dirac approximation of the reverse transition, based on the conditional mean, is poor at coarse time discretizations.
Approach
Train a noise-conditioned generator to sample from the full posterior p(x0|xt) by minimizing a conditional generalized energy score (or kernel score) instead of MSE regression; the lambda and beta hyperparameters interpolate to standard diffusion loss, and the generator's samples are plugged into DDIM-style updates.
Key results
- On conditional image generation, distributional models with lambda=1, beta<2 outperform standard diffusion models at few denoising steps, with the authors reporting a ~3x FID gap to diffusion in the conditional few-step regime versus ~1.5x unconditional.
- Distributional models with lambda in (0,1) remain competitive at many steps, and beta<2, lambda=0 gives comparable or slightly better FID than MSE diffusion.
- On LIBERO-Long, lambda>0 gives best success rates with 2 and 16 diffusion steps, while lambda=0 is best at 50 steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a distributional model with λ=1 and β<2 instead of a standard diffusion model | Image generation · CIFAR-10, CelebA, LSUN Bedrooms, and latent CelebA-HQ | FID | Helped |
| Use a distributional model with λ=1 and β<2 instead of a standard diffusion model | Image generation · CIFAR-10, CelebA, LSUN Bedrooms, and latent CelebA-HQ | FID | Hurt |
| Use λ∈(0,1) rather than λ=1 in the distributional model (ablation) | Image generation · CIFAR-10, CelebA, LSUN Bedrooms, and latent CelebA-HQ | FID | Helped |
| Use β<2 and λ=0 instead of the classical diffusion objective (ablation) | Diffusion models · Image generation | FID | Mixed |
| Use a distributional model with λ=1 and β<2 instead of a standard diffusion model | Image generation · Conditional vs. unconditional image generation | FID: 1.5x; 3x | Mixed |
| Use a distributional model with λ=1 and β=0.1 instead of a standard diffusion model | 2D diffusion model · Two-Gaussian mixture | squared MMD | Helped |
| Use a distributional model with λ=1 and β=0.1 instead of a standard diffusion model | 2D diffusion model · Two-Gaussian mixture | posterior standard deviation | Helped |
| Use a distributional model with λ>0 instead of λ=0 (ablation) | Diffusion policy · Libero-Long | success rate | Helped |
| Use λ>0 instead of λ=0 (ablation) | Diffusion policy · Libero-Long | success rate | Hurt |
| Use a kernel diffusion loss with λ=1 for few steps and λ<1 for more steps (ablation) | Distributional diffusion model · CIFAR-10 conditional generation | FID | Mixed |
| Use the proposed distributional approach instead of DPM-solver++ | Image generation · Image experiments | FID | Helped |
| Use the proposed distributional approach instead of moment-matching distillation | Image generation · Image experiments | FID | No clear effect |
Surprising findings
- Diffusion models can be trained with scoring-rule losses other than MSE (beta<2, lambda=0) with comparable or slightly better FID than standard MSE diffusion.
- Gaussian analysis shows lambda<1 underestimates target variance by a factor f(lambda,beta), yet few-step performance still improves.
Limitations
- Performance degrades at large numbers of steps when lambda=1 due to error accumulation from posterior-sample noise.
- Generator architecture only concatenates the noise to x_t channel-wise; other architectures are unexplored.
- Kernel scores require choosing lambda and beta, with best values varying by setting and step count.
Experimental setup
Datasets: CIFAR-10 CelebA LSUN Bedrooms CelebA-HQ LIBERO
Benchmarks: CIFAR-10 CelebA LSUN Bedrooms CelebA-HQ latent LIBERO-Long
Metrics: FID squared MMD success rate
Baselines: standard diffusion model (MSE regression) DPM-Solver++ moment-matching distillation DDIM
Builds on: DDIM flow matching energy score DPM-Solver++ engression
Closest papers by meaning
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.91)
- Particle Denoising Diffusion Sampler (ICML 2024 · similarity 0.90)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.90)
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.90)
- Dimension-free Score Matching and Time Bootstrapping for Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Diffusion Path Samplers via Sequential Monte Carlo (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.