Research map / Image generation / Fast diffusion sampling
A Variational Perspective on Solving Inverse Problems with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RED-diff, a variational (KL-minimization) sampler that turns diffusion-based inverse problem solving into stochastic optimization over a regularization-by-denoising loss with SNR-based timestep weighting.
Problem
Sampling the posterior p(x0|y) for inverse problems with pretrained diffusion priors is intractable because the posterior score over diffusion steps must be approximated, and existing guidance methods (DPS, ΠGDM) rely on crude unimodal approximations and costly score backpropagation.
Approach
Derive a variational KL objective whose regularizer is a weighted score-matching loss over the whole diffusion trajectory, which reduces (with a Dirac variational posterior and omega(0)=0) to a gradient involving only denoiser residuals (no Jacobian); the weighting is chosen via inverse SNR, and sampling becomes optimizing an image with off-the-shelf optimizers like Adam.
Key results
- On ImageNet inpainting, RED-diff reports PSNR 23.29 dB, SSIM 0.87, KID 0.86 versus DPS 21.27 dB and ΠGDM 20.30 dB, with 0.05 s per step versus 0.23 s for DPS.
- On ImageNet 4x superresolution, RED-diff achieves the best PSNR (25.95 dB) but not the best LPIPS (0.25 vs 0.15 for ΠGDM).
- On nonlinear HDR, RED-diff reaches 25.23 dB PSNR versus 7.94 dB for DPS; on deblurring it reaches 45.00 dB versus 6.4 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RED-diff instead of DPS | ImageNet inpainting · ImageNet inpainting | PSNR (dB): 21.27 -> 23.29 | Helped |
| Use RED-diff instead of DPS | Phase Retrieval · Phase Retrieval | KID: 93.2 -> 114.0 | Hurt |
| Use RED-diff instead of DPS | Phase Retrieval · Phase Retrieval | PSNR (dB): 9.99 -> 10.53 | Helped |
| Use RED-diff instead of DPS | HDR · HDR | PSNR (dB): 7.94 -> 25.23 | Helped |
| Use RED-diff instead of DPS | Deblurring · Deblurring | PSNR (dB): 6.4 -> 45.00 | Helped |
| Use RED-diff instead of the strongest baseline | ImageNet 4x superresolution · ImageNet 4x superresolution | LPIPS: 0.15 -> 0.25 | Hurt |
| Use RED-diff instead of CSGM | Compressed sensing MRI · fastMRI brain, R = 4 | PSNR (dB): 36.3 -> 37.1 | Helped |
| Use RED-diff instead of DPS | Noisy Palette inpainting · Noisy Palette inpainting, σy=0.1 | PSNR (dB): 16.98 -> 18.92 | Helped |
| Use 100 optimization steps instead of 10 (ablation) | RED-diff · ImageNet inpainting | PSNR (dB): 18.83 -> 23.13 | Helped |
| Use 100 optimization steps instead of 10 (ablation) | RED-diff · ImageNet inpainting | KID: 22.05 -> 1.93 | Helped |
| Use 1000 optimization steps instead of 100 (ablation) | RED-diff · ImageNet inpainting | PSNR (dB): 23.13 -> 23.86 | Helped |
| Use 1000 optimization steps instead of 100 (ablation) | RED-diff · ImageNet inpainting | KID: 1.93 -> 2.17 | Hurt |
Surprising findings
- Descending timestep order (T to 0) with one denoiser per step outperforms random, ascending, and mini-batch timestep sampling, and batch sampling over many denoisers smears fine texture.
- Adding more optimization epochs gives negligible quality improvement, and SGD with momentum performs about as well as Adam.
Limitations
- Mode-seeking nature gives limited sample diversity, promoting MAP-like solutions.
- Lower perceptual quality (LPIPS) than baselines on 4x superresolution, showing a fidelity-perceptual trade-off.
- Sensitive to the choice of weighting and learning rate; batch timestep sampling degrades details.
Experimental setup
Datasets: ImageNet Palette masks fastMRI brain Mridata knee
Benchmarks: ImageNet 1k validation subset (inpainting, nonlinear tasks, superresolution) fastMRI brain Mridata knee
Metrics: PSNR SSIM KID LPIPS top-1 classifier accuracy time per step max batch size
Baselines: DPS ΠGDM DDRM CSGM-Langevin
Builds on: DPS ΠGDM DDRM regularization by denoising (RED) DDPM
Closest papers by meaning
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.93)
- Diffusion Priors for Variational Likelihood Estimation and Image Denoising (NeurIPS 2024 · similarity 0.92)
- Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems (NeurIPS 2024 · similarity 0.92)
- Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors (NeurIPS 2024 · similarity 0.92)
- A Diffusion Model with State Estimation for Degradation-Blind Inverse Imaging (AAAI 2024 · similarity 0.92)
- Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models (ICML 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.