Research map / Image generation / Fast diffusion sampling
Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues that DPS behaves more like maximum a posterior (MAP) than conditional score estimation, and proposes DMAP (multi-step gradient ascent with sphere projection) plus a lightweight ControlNet conditional score estimator to boost DPS on 512x512 ImageNet inverse problems.
Problem
Diffusion Posterior Sampling (DPS) is widely explained as approximating the conditional score for solving inverse problems, but this explanation is questioned; the paper examines whether DPS actually estimates the conditional score on 512x512 images.
Approach
Shows DPS's implied conditional score has large error, large nonzero mean and low diversity, hypothesizes DPS approximates MAP, and improves it with multi-step gradient ascent on the likelihood followed by projection onto the sphere where the Gaussian transition concentrates, initialized from a cheaply trained ControlNet conditional score estimator.
Key results
- DMAP (full) reaches PSNR 23.52, LPIPS 0.3494, FID 39.56 on SR x8 versus DPS at 22.33 / 0.4137 / 58.48.
- DPS + CSE (n=100) improves Gaussian deblur FID from 59.31 to 23.39 and LPIPS from 0.4267 to 0.2306.
- DPS with zeta=4.8 has score mean about 5.86 versus about 0.40 for the unconditional score, and per-pixel std 0.0453 versus 0.3939 for StableSR (with k=50 samples).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace DPS with multi-step gradient ascent and spherical projection (DMAP) | DPS · ImageNet 512 SR×8 | FID: 58.48 -> 39.56 | Helped |
| Replace DPS with multi-step gradient ascent and spherical projection (DMAP) | DPS · ImageNet 512 Gaussian deblur | PSNR: 23.05 -> 24.86 | Helped |
| Replace DPS with multi-step gradient ascent and spherical projection (DMAP) | DPS · ImageNet 512 non-linear deblur | FID: 62.10 -> 39.93 | Helped |
| Add CSE trained on 100 ImageNet images | DPS · ImageNet 512 SR×8 | FID: 58.48 -> 44.59 | Helped |
| Add CSE trained on 100 ImageNet images | DPS · ImageNet 512 Gaussian deblur | FID: 59.31 -> 23.39 | Helped |
| Add CSE trained on 100 ImageNet images | DPS · ImageNet 512 non-linear deblur | FID: 62.10 -> 40.25 | Helped |
| Add CSE trained on 100 ImageNet images | DMAP · ImageNet 512 Gaussian deblur | PSNR: 24.86 -> 26.17 | Helped |
| Increase CSE training data from 25 to 100 ImageNet images (ablation) | DPS + CSE · ImageNet 512 SR×8 | FID: 75.64 -> 44.59 | Helped |
| Increase CSE training data from 100 to 1000 ImageNet images (ablation) | DPS + CSE · ImageNet 512 SR×8 | PSNR: 22.98 -> 23.35 | Helped |
| Train CSE on self-generated images instead of ImageNet images (ablation) | DPS + CSE · ImageNet 512 SR×8 | LPIPS: 0.4137 -> 0.3082 | Helped |
| Increase CSE training time from 0 to 2 GPU hours (ablation) | DPS + CSE · ImageNet 512 SR×8 | PSNR: 22.33 -> 23.04 | Helped |
| Increase DMAP gradient steps to K=2 while reducing diffusion steps to T=50 (ablation) | DMAP · ImageNet 512 SR×8 | LPIPS: 0.4763 -> 0.4591 | Helped |
Surprising findings
- The DPS conditional score estimate is further from StableSR's score than the unconditional score is, and a larger score error correlates with better image quality for DPS.
- A conditional score estimator alone (CSE without DPS) performs poorly, but combined with DPS it substantially improves results.
Limitations
- The CSE relies on ControlNet, which is not sample- or compute-efficient; the authors say few-shot improvement is an open direction.
- The strict theoretical link between original DPS and MAP is left open; the MAP derivation relies on high-dimensional asymptotics and assumptions that hold approximately.
- DMAP (full) costs about 3x the runtime of DPS (517 s vs 162 s).
Experimental setup
Datasets: ImageNet validation (first 1000 images, 512x512) ImageNet training subsets (25, 50, 100, 1000, 100000 images) LSUN bedroom (first 100 images) Stable Diffusion 2.0 self-generated images
Benchmarks: ImageNet 512 (SR x8, Gaussian deblur, non-linear deblur, inpainting) LSUN bedroom segmentation
Metrics: PSNR LPIPS FID KID MIoU score error score mean per-pixel standard deviation
Baselines: DPS PSLD ReSample FreeDOM DSG ControlNet
Builds on: Diffusion Posterior Sampling (DPS) DSG ControlNet DDPM FreeDOM
Closest papers by meaning
- Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction (NeurIPS 2024 · similarity 0.91)
- Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors (NeurIPS 2024 · similarity 0.91)
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.89)
- SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems (ICML 2025 · similarity 0.89)
- Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps (ICLR 2024 · similarity 0.89)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.