Research map / Image generation / Fast diffusion sampling
Risk-Sensitive Diffusion: Robustly Optimizing Diffusion Models with Noisy Samples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes risk-sensitive SDEs, parameterized by per-sample risk vectors, that make diffusion training on noisy samples match the clean marginal distribution within a stability interval, with closed-form coefficients for Gaussian and Cauchy noise.
Problem
Diffusion models degrade when trained on noisy non-image data (e.g., tabular and time series) where each sample comes with a risk/quality vector, since noisy samples shift the marginal distributions along the diffusion process.
Approach
Define a risk-sensitive SDE whose drift and diffusion coefficients depend on the risk vector, chosen so that the noisy sample's marginal matches the clean marginal at the stability interval (exact for isotropic Gaussian noise); for non-Gaussian noise, minimize a perturbation instability measure based on cumulant-generating functions.
Key results
- On Telemonitoring with VE SDE, risk-sensitive VE SDE reaches Wasserstein distance 1.582 versus 3.159 for the best risk-conditional baseline.
- On MIMIC-III with VP SDE, risk-sensitive VP SDE reaches Wasserstein distance 1.625 versus 5.637 for the risk-conditional baseline.
- On CIFAR-10 with 40% noisy images, risk-sensitive diffusion obtains FID 6.97 versus 8.53 for Unbiased Diffusion Model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use risk-sensitive VE SDE instead of risk-conditional VE SDE | VE SDE · MIMIC-III | Wasserstein distance: 5.926 -> 1.865 | Helped |
| Use risk-sensitive VE SDE instead of risk-conditional VE SDE | VE SDE · WARDS | Wasserstein distance: 5.951 -> 2.513 | Helped |
| Use risk-sensitive VE SDE instead of risk-conditional VE SDE | VE SDE · Abalone | Wasserstein distance: 0.612 -> 0.089 | Helped |
| Use risk-sensitive VE SDE instead of risk-conditional VE SDE | VE SDE · Telemonitoring | Wasserstein distance: 3.159 -> 1.582 | Helped |
| Use risk-sensitive VE SDE instead of risk-conditional VE SDE | VE SDE · Mushroom | Wasserstein distance: 4.101 -> 0.713 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · MIMIC-III | Wasserstein distance: 5.637 -> 1.625 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · WARDS | Wasserstein distance: 6.292 -> 2.584 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · Abalone | Wasserstein distance: 0.585 -> 0.077 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · Telemonitoring | Wasserstein distance: 3.785 -> 1.462 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · Mushroom | Wasserstein distance: 4.850 -> 0.852 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · Abalone | MMD: 0.00766 -> 0.00198 | Helped |
| Use risk-sensitive VP SDE instead of risk-conditional VP SDE | VP SDE · Telemonitoring | MMD: 0.01267 -> 0.00717 | Helped |
Surprising findings
- Assuming Gaussian perturbation works well in practice even when the true noise is Cauchy or mis-specified.
- Risk-conditional diffusion models produce biased sampling distributions that underrepresent high-noise regions.
Limitations
- Exact stability is only achievable for Gaussian noise; non-Gaussian noise yields only minimal instability.
- Risk vectors must be available or estimated, e.g., via Gaussian process variances or KNN deviations; tabular noise is artificially created by masking.
- Code is not yet released; the authors say it will be released after acceptance.
Experimental setup
Datasets: MIMIC-III WARDS Abalone Telemonitoring Mushroom CIFAR-10
Benchmarks: MIMIC-III WARDS Abalone Telemonitoring Mushroom CIFAR-10
Metrics: Wasserstein distance MMD PRD curve FID
Baselines: Standard VE SDE Standard VP SDE VE/VP SDE w/ Risk Regressor VE/VP SDE w/ Risk Variable VE/VP SDE w/ Risk Conditional Ambient Diffusion Unbiased Diffusion Model
Builds on: score-based generative models VP SDE VE SDE classifier-free guidance
Closest papers by meaning
- Heavy-Tailed Diffusion with Denoising Levy Probabilistic Models (ICLR 2025 · similarity 0.87)
- Robust Diffusion Models via Divergence-Induced Weighted Denoising (NeurIPS 2026 · similarity 0.87)
- Training-Free Safe Denoisers for Safe Use of Diffusion Models (NeurIPS 2025 · similarity 0.87)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.87)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.87)
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.