Research map / Image generation / Fast diffusion sampling
Heavy-Tailed Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Repurposes diffusion models with multivariate Student-t perturbation kernels and a gamma-power divergence objective, yielding t-EDM and t-Flow that better capture heavy tails on HRRR weather data.
Problem
Standard Gaussian-prior diffusion and flow models fail to capture heavy-tailed behavior and rare extreme events in high-dimensional data such as weather fields.
Approach
Replace the Gaussian perturbation kernel, forward/reverse posteriors, and KL objective of diffusion models with multivariate Student-t counterparts, trained via a closed-form gamma-power divergence with gamma = -2/(nu+d); the degrees-of-freedom nu gives controllable tail heaviness and reduces to Gaussian diffusion as nu grows.
Key results
- On unconditional VIL generation (HRRR), t-EDM with nu=3 reaches KS 0.114 on test versus 0.991 for EDM.
- For conditional VIL prediction, t-EDM (nu=3) improves KS to 0.419 vs 0.715 for EDM, with CRPS 1.649 vs 1.696.
- t-Flow (nu=3) lowers VIL test KS to 0.275 compared with 0.704 for Gaussian flow.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Gaussian-prior EDM with t-EDM (ν=3) | EDM · HRRR VIL (Test) | KS: 0.522 -> 0.114 | Helped |
| replace Gaussian-prior EDM with t-EDM (ν=3) | EDM · HRRR w20 (Test) | KS: 0.648 -> 0.286 | Helped |
| replace Gaussian Flow with t-Flow (ν=3) | Gaussian Flow · HRRR VIL (Test) | KS: 0.704 -> 0.275 | Helped |
| replace Gaussian Flow with t-Flow (ν=9) | Gaussian Flow · HRRR w20 (Test) | KS: 0.384 -> 0.289 | Helped |
| replace EDM with t-EDM (ν=5) | EDM · HRRR VIL (Test) | CRPS: 1.696 -> 1.609 | Helped |
| replace EDM with t-EDM (ν=5) | EDM · HRRR VIL (Test) | SSR: 0.203 -> 0.305 | Helped |
| replace EDM with t-EDM (ν=3) | EDM · HRRR w20 (Test) | CRPS: 0.304 -> 0.295 | Helped |
| replace EDM with t-EDM (ν=3) | EDM · HRRR w20 (Test) | SSR: 0.865 -> 1.045 | Helped |
| replace EDM with t-EDM (ν=3) | EDM · HRRR w20 (Test) | RMSE: 0.664 -> 0.734 | Hurt |
| use t-EDM with ν=5 instead of ν=3 (ablation) | t-EDM · HRRR VIL (Test) | KS: 0.114 -> 0.888 | Hurt |
| use t-EDM with ν=7 instead of ν=3 (ablation) | t-EDM · HRRR w20 (Test) | KS: 0.286 -> 0.622 | Hurt |
Surprising findings
- Standard EDM fails to capture tails even with noise conditioning, and the optimal nu differs across channels (VIL vs w20).
Limitations
- Heteroskedastic reverse-posterior scale is not modeled; only the mean is parameterized.
- Optimal degrees of freedom nu varies by channel and must be tuned rather than learned.
Experimental setup
Datasets: HRRR
Benchmarks: HRRR
Metrics: Kurtosis Ratio Skewness Ratio Kolmogorov-Smirnov statistic CRPS RMSE skill-spread ratio
Baselines: EDM EDM + INC EDM + PCP Gaussian Flow
Builds on: EDM straight-line flows (stochastic interpolants) gamma-power divergence DDPM
Closest papers by meaning
- Heavy-Tailed Diffusion with Denoising Levy Probabilistic Models (ICLR 2025 · similarity 0.92)
- Heavy-Tailed Flow Matching via Random Clocks (NeurIPS 2026 · similarity 0.89)
- Robust Diffusion Models via Divergence-Induced Weighted Denoising (NeurIPS 2026 · similarity 0.87)
- Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.86)
- Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs (ICLR 2025 · similarity 0.86)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.