Research map / Image generation / Fast diffusion sampling

Robust Diffusion Models via Divergence-Induced Weighted Denoising

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Denoising diffusion probabilistic models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes replacing the diffusion MSE denoising loss with f-divergence-induced transformations (Hellinger, negative exponential divergence), giving divergence-induced influence weights that improve robustness to training-data contamination on CIFAR-10/100.

Problem

Standard diffusion training weights every sample equally under MSE, so corrupted or atypical training images degrade generative quality.

Approach

Under Gaussian DDPM reverse kernels the per-step likelihood ratio is lognormal with a scalar mismatch m_t, so each local f-divergence reduces to a 1-D function h_G(m_t); the gradient is reweighted by h_G'(m_t), giving bounded-influence weights (HD: exp(-m/4); NED via quadrature) that suppress high-error samples.

Key results

• On CIFAR-10 at 30% contamination, NED reduces FID from 93.0 (KL) to 77.5 (mean over 3 seeds).
• In a single-seed comparison at 30% contamination, HD and NED reach FID 56.2 and 55.1 vs 94.0 (Huber) and 90.5 (clipped MSE).
• On CIFAR-100 at 30% contamination, NED achieves FID 75.3 vs 98.0 for KL (single seed).

Surprising findings

• On clean CIFAR-10 the KL objective has much higher cross-seed FID variance (12.3) than HD or NED, and NED at 16 quadrature nodes (FID 25.7) beats 64 nodes (29.7) in a single-seed ablation.
• HD gives essentially no improvement over KL at 30% contamination on CIFAR-10 (91.9 vs 93.0), and HD is worse than KL on CIFAR-100 at 30%.

Limitations

• Local conditional objective is a surrogate, not a proven upper bound on the marginal divergence for non-KL f-divergences.
• Small 35M model, short 100K-step training, and FID from 5K samples; large cross-seed variance, with robustness claims lacking formal bounds.
• Robust baselines (Huber, clipped MSE) used default thresholds without tuning; flow matching extension untested.

Experimental setup

Datasets: CIFAR-10 CIFAR-100

Benchmarks: CIFAR-10 CIFAR-100

Metrics: FID

Baselines: KL (standard DDPM MSE) Huber loss clipped MSE

Builds on: DDPM f-divergence Huber loss M-estimation

Closest papers by meaning

• Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.90)
• O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.90)
• Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.90)
• Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference (NeurIPS 2024 · similarity 0.89)
• EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.89)
• Efficient Denoising Diffusion via Probabilistic Masking (ICML 2024 · similarity 0.89)