Research map / Image generation / Fast diffusion sampling
Consistency Posterior Sampling for Diverse Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes posterior sampling via Langevin dynamics in the noise space of a pre-trained consistency model, enabling cheap accumulation of diverse posterior samples for noisy linear and nonlinear image restoration.
Problem
Generating diverse posterior samples for inverse problems with generative priors is expensive because existing diffusion-based methods re-run the full sampling chain for each new sample.
Approach
Run Euler-Maruyama Langevin dynamics directly in the noise space of a one-to-one consistency-model (or flow) mapping, warm-started by Adam minimization of the measurement loss, so each new posterior sample costs only a few function evaluations; a TV bound ties the sampler error to the prior approximation error.
Key results
- On LSUN-Bedroom 8x super-resolution, the 1-step method reaches PSNR 20.4 and FID 71.1, competitive with DPS-DM (20.4, 67.7) while CM-based baselines reach PSNR around 10.5-10.7.
- On 10% inpainting, the 1-step method reaches PSNR 23.8 and LPIPS 0.358, outperforming CM baselines DPS-CM (19.9) and LGD-CM (19.9) in PSNR.
- Diversity Score on 8x super-resolution rises to 3.01 for the 1-step method versus 2.14 for DPS-DM and 2.35 for LGD-DM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use noise-space Langevin sampling with a 1-step CM mapping rather than DPS-DM | posterior sampling on LSUN-Bedroom · 10% inpainting | Diversity Score: 2.33 -> 3.15 | Helped |
| Use noise-space Langevin sampling with a 1-step CM mapping rather than DPS-DM | posterior sampling on LSUN-Bedroom · 8× super-resolution | Diversity Score: 2.14 -> 3.01 | Helped |
| Use noise-space Langevin sampling with a 1-step CM mapping rather than DPS-DM | posterior sampling on LSUN-Bedroom · Gaussian deblur | Diversity Score: 2.10 -> 3.26 | Helped |
| Use noise-space Langevin sampling with a 1-step CM mapping rather than DPS-DM | posterior sampling on LSUN-Bedroom · phase retrieval | Diversity Score: 2.42 -> 3.08 | Helped |
| Use noise-space Langevin sampling with a 1-step CM mapping rather than DPS-DM | posterior sampling on LSUN-Bedroom · nonlinear deblur | Diversity Score: 2.22 -> 2.80 | Helped |
| Use noise-space Langevin sampling with a 1-step CM mapping rather than DPS-DM | posterior sampling on LSUN-Bedroom · HDR reconstruction | Diversity Score: 2.25 -> 3.09 | Helped |
| Use 1-step rather than 2-step CM mapping (ablation) | CPS · LSUN-Bedroom Gaussian deblur | PSNR: 21.3 -> 22.4 | Helped |
| Use 1-step rather than 2-step CM mapping (ablation) | CPS · ImageNet 4× super-resolution | PSNR: 18.1 -> 16.9 | Hurt |
| Increase warm-start optimization iterations K (ablation) | CPS · image restoration | FID: 95 to below 82.5 | Helped |
| Increase warm-start optimization iterations K (ablation) | CPS · image restoration | Diversity Score | Helped |
| Vary the number of EM sampling iterations N over 50 steps (ablation) | CPS · 8× super-resolution and nonlinear deblurring | PSNR | No clear effect |
Surprising findings
- Diversity gains come with some fidelity loss: the authors note low visual quality in some posterior samples, attributed to the relatively poor prior approximation by consistency models.
Limitations
- Low visual quality in some posterior samples due to imperfect consistency-model prior.
- Warm-start and gradient computation through the generator add overhead; gradients require backpropagation through the CM.
- Evaluated only on 100 validation images per dataset and on 64x64 and 256x256 resolutions.
Experimental setup
Datasets: LSUN-Bedroom ImageNet 64x64
Benchmarks: LSUN-Bedroom 256x256 ImageNet 64x64
Metrics: PSNR SSIM LPIPS FID Diversity Score CLIP cosine similarity
Baselines: DPS LGD MPGD CMEdit DPS-CM LGD-CM MPGD-CM
Builds on: consistency models diffusion posterior sampling langevin dynamics LGD MPGD
Closest papers by meaning
- Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction (AAAI 2024 · similarity 0.88)
- Restoration by Generation with Constrained Priors (CVPR 2024 · similarity 0.88)
- NoiseCtrl: A Sampling-Algorithm-Agnostic Conditional Generation Method for Diffusion Models (CVPR 2025 · similarity 0.88)
- Improving Diffusion Posterior Samplers with Lagged Temporal Corrections for Image Restoration (NeurIPS 2026 · similarity 0.88)
- Diffusion Priors for Variational Likelihood Estimation and Image Denoising (NeurIPS 2024 · similarity 0.88)
- DEFT: Efficient Fine-tuning of Diffusion Models by Learning the Generalised $h$-transform (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.