Research map / Image generation / Fast diffusion sampling
Heavy-Tailed Diffusion with Denoising Levy Probabilistic Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Denoising Lévy Probabilistic Models (DLPM), a DDPM variant that replaces Gaussian noise with isotropic α-stable noise via a Gaussian scale-mixture decomposition, plus a deterministic sampler DLIM.
Problem
Diffusion models use Gaussian noise, which limits coverage of heavy tails and mode collapse on unbalanced data; the existing heavy-tailed SDE approach (LIM) is mathematically complex, has an ill-posed squared loss with infinite-variance noise, and is hard to adapt to noise schedules.
Approach
Write α-stable noise as sqrt(A) times Gaussian (A positive stable), so conditioned on A the forward and backward kernels are Gaussian; train a DDPM-style denoiser with a square-root (r=1/2) loss averaged over A, reusing standard DDPM architectures and code with minor changes.
Key results
- Generated images after 25 steps achieve FID 2.81 on MNIST and 20.69 on CIFAR10_LT, versus 45.72 and 30.17 for LIM-ODE with clipping.
- On MNIST with alpha=1.7, DLIM matches DLPM sample quality with 40 times fewer diffusion steps.
- On 2D data, DLPM with alpha<2 improves tail coverage (lower MSLE) and F1 score over the Gaussian case, and consistently outperforms LIM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DLIM instead of LIM-ODE with clipping at 25 sampling steps | Image generation · MNIST | FID: 45.72 -> 2.81 | Helped |
| Use DLIM instead of LIM-ODE with clipping at 25 sampling steps | Image generation · CIFAR10_LT | FID: 30.17 -> 20.69 | Helped |
| Use heavy-tailed noise with lower α | DLPM · Heavy-tailed 2D stable data | MSLE | Helped |
| Use DLPM with α < 2 instead of the Gaussian case | DLPM · Unbalanced nine-Gaussian mixture | FPR | Helped |
| Use deterministic rather than stochastic sampling | Diffusion image generation · MNIST and CIFAR10_LT | sample quality | Helped |
Surprising findings
- LIM-ODE performs worse than Gaussian diffusion at 25 reverse steps, despite lower alpha usually giving lower FID in the DLPM table.
- LIM performance deteriorates significantly when clipping is not used, raising doubts about LIM's suitability for heavy tails.
Limitations
- DLPM_5 (median-of-means with M=5) costs roughly 5 times the training run time.
- Cauchy (alpha=1) DLIM variant is not experimentally explored.
- Heavy-tailed noise can cause training instability; LIM needed clipping, and the squared loss is not finite for alpha-stable noise.
Experimental setup
Datasets: MNIST CIFAR10_LT two-dimensional synthetic heavy-tailed and Gaussian-mixture data
Benchmarks: MNIST CIFAR10_LT
Metrics: FID F_1 score (precision-recall harmonic mean) MSLE tail quantile error
Baselines: LIM LIM-ODE DDPM DDIM
Builds on: DDPM DDIM Lévy-Itô model (LIM) score-based SDE
Closest papers by meaning
- Heavy-Tailed Diffusion Models (ICLR 2025 · similarity 0.92)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.90)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.89)
- Robust Diffusion Models via Divergence-Induced Weighted Denoising (NeurIPS 2026 · similarity 0.88)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.88)
- Beyond Scores: Proximal Diffusion Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.