Research map / Image generation / Fast diffusion sampling

Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Models

ICLR 2024 · Image generation › Fast diffusion sampling · approach: Denoising diffusion probabilistic models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows theoretically that Gaussian backward denoising in diffusion models is an expressive bottleneck with unbounded errors, and proposes soft mixture denoising (SMD), a hypernetwork-based continuous relaxation of Gaussian mixtures that improves DDPM, ADM and LDM FID especially at few sampling steps.

Problem

Diffusion models use a single Gaussian per backward denoising step, which cannot fit multimodal posteriors q(x_{t-1}|x_t); existing universal-approximation guarantees assume bounded score errors, which the authors argue is too strong.

Approach

Replace the Gaussian backward kernel with a soft (continuous-latent) Gaussian mixture: a latent z_t from a transformed Gaussian noise selects per-sample Gaussian parameters via a hypernetwork f_phi, yielding a tractable upper bound on the NLL that reduces to a noise-prediction loss with Monte Carlo sampling over eta.

Key results

• On CIFAR-10 32x32 with T=1000, DDPM FID drops from 3.78 to 3.13 with SMD, and ADM from 2.98 to 2.55.
• On LSUN-Church 256x256 LDM FID improves from 5.86 to 5.21 and on CelebA-HQ 256x256 from 6.13 to 5.48 (T=1000).
• SMD improves LDM more at few backward iterations (e.g., T=200), and sampling multiple eta at training further lowers FID (6.5% for 5 samples on LSUN-Conference).

Measured effects reported in the paper

Surprising findings

• Theory claims the local and global denoising errors of standard diffusion models can be arbitrarily large for Gaussian mixture data, undermining bounded score-error assumptions in prior guarantees.

Limitations

• Theoretical guarantees assume globally optimized networks and rely on a constructed counterexample rather than characterizing typical data.
• Evaluation is limited to FID on image datasets with DDPM/ADM/LDM backbones; no comparison with other recent samplers or large text-to-image models.
• Extra sampling of eta during training trades run time for quality.

Experimental setup

Datasets: CIFAR-10 LSUN-Conference LSUN-Church CelebA-HQ

Benchmarks: CIFAR-10 LSUN-Conference LSUN-Church CelebA-HQ

Metrics: FID

Baselines: DDPM ADM LDM DDIM

Builds on: DDPM DDIM ADM Latent Diffusion Model

Closest papers by meaning

• Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy (NeurIPS 2024 · similarity 0.89)
• Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models (ICLR 2025 · similarity 0.89)
• A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.89)
• Residual Denoising Diffusion Models (CVPR 2024 · similarity 0.88)
• AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.88)
• UDPM: Upsampling Diffusion Probabilistic Models (NeurIPS 2024 · similarity 0.88)