Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that zero-shot diffusion inverse-problem solvers (DPS, ΠGDM, DDNM, DiffPIR) can be unified as isotropic Gaussian approximations of the denoising posterior, and proposes learning posterior covariance via MLE (converted from reverse variances, Monte Carlo, or DWT-domain diagonal) to improve reconstruction without hyperparameter tuning.
Problem
Zero-shot solvers for noisy linear inverse problems with pre-trained unconditional diffusion models rely on hand-crafted isotropic covariances for the denoising posterior, which is a poor approximation of the conditional posterior mean and requires hyperparameter tuning.
Approach
Unifies recent zero-shot solvers as Gaussian approximations to p(x0|xt) with isotropic covariance, then replaces the hand-crafted covariance with a learned/MLE-optimal one: converted from pre-trained DDPM reverse variances, estimated by Monte Carlo, or learned as a diagonal in an orthonormal (DWT) basis.
Key results
- On FFHQ, the Convert method achieves SSIM 0.9279 / LPIPS 0.0794 / FID 25.90 on random inpainting, outperforming DPS and ΠGDM under the same sampler.
- DWT-Var outperforms optimally tuned DiffPIR by a significant margin on Type II guidance without hyperparameter tuning.
- Plugging optimal covariance into ΠGDM at last sampling steps yields large gains, e.g., ImageNet inpainting FID improving from 141.80 to 113.39-ish range under adaptive-weight setting (Convert: 33.46 vs ΠGDM 141.80).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use reverse-variance conversion for posterior covariance | Type I guidance on FFHQ · Inpainting (random), SSIM | SSIM: 0.8891 -> 0.9279 | Helped |
| Use DWT-domain posterior variance | Type I guidance on FFHQ · Gaussian deblurring, SSIM | SSIM: 0.7890 -> 0.7968 | Helped |
| Use DWT-domain posterior variance | Type I guidance on FFHQ · Motion deblurring, SSIM | SSIM: 0.7543 -> 0.7677 | Helped |
| Use DWT-domain posterior variance | Type I guidance on FFHQ · Super-resolution (4×), SSIM | SSIM: 0.7719 -> 0.8025 | Helped |
| Use reverse-variance conversion for posterior covariance | Type I guidance on ImageNet · Inpainting (random), SSIM | SSIM: 0.8623 -> 0.8559 | Hurt |
| Use analytically estimated posterior covariance | Type I guidance on ImageNet · Gaussian deblurring, SSIM | SSIM: 0.5946 -> 0.6009 | Helped |
| Use reverse-variance conversion for posterior covariance | Type I guidance on ImageNet · Motion deblurring, SSIM | SSIM: 0.5534 -> 0.5634 | Helped |
| Use analytically estimated posterior covariance | Type I guidance on ImageNet · Super-resolution (4×), SSIM | SSIM: 0.5925 -> 0.5958 | Helped |
| Replace the last sampling steps with optimal posterior variances | ΠGDM adaptive-weight guidance on FFHQ · Inpainting (random), SSIM | SSIM: 0.7078 -> 0.9241 | Helped |
| Replace the last sampling steps with optimal posterior variances | ΠGDM adaptive-weight guidance on FFHQ · Gaussian deblurring, SSIM | SSIM: 0.7221 -> 0.7790 | Helped |
| Replace the last sampling steps with optimal posterior variances | ΠGDM adaptive-weight guidance on FFHQ · Motion deblurring, SSIM | SSIM: 0.6977 -> 0.7331 | Helped |
| Replace the last sampling steps with optimal posterior variances | ΠGDM adaptive-weight guidance on FFHQ · Super-resolution (4×), SSIM | SSIM: 0.7205 -> 0.7632 | Helped |
Surprising findings
- Type II guidance methods performed poorly with deterministic samplers and needed stochastic Heun sampling.
- Spatial posterior variance from the converted reverse variances was unstable at high noise levels, so the method was only applied at low-noise final steps.
Limitations
- Diagonal covariance constraint means the optimal covariance cannot be matched exactly even for a perfectly trained model.
- Spatial variance is unstable at high noise levels, requiring heuristic switching to other covariances.
- Evaluated only on FFHQ and ImageNet at 256x256 with linear inverse problems and fixed noise level sigma=0.05.
Experimental setup
Datasets: FFHQ 256x256 ImageNet 256x256
Benchmarks: FFHQ 256x256 inpainting, deblurring, super-resolution ImageNet 256x256 inpainting, deblurring, super-resolution
Metrics: SSIM LPIPS FID
Baselines: DPS ΠGDM DiffPIR DDNM TMPD
Builds on: DPS ΠGDM DDNM DiffPIR DDPM
Closest papers by meaning
- Exact Posterior Score Estimation for Solving Linear Inverse Problems (NeurIPS 2026 · similarity 0.91)
- Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems (NeurIPS 2024 · similarity 0.91)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.90)
- InvFusion: Bridging Supervised and Zero-shot Diffusion for Inverse Problems (NeurIPS 2025 · similarity 0.90)
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.90)
- Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.