Research map / Image generation / Fast diffusion sampling
Learning Diffusion Priors from Observations by Expectation Maximization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiEM, an expectation-maximization scheme that trains proper diffusion priors from noisy, incomplete linear observations only, paired with a moment-matching posterior sampler (MMPS).
Problem
Training diffusion priors when only incomplete and noisy linear observations are available, with no clean latent samples.
Approach
Alternate posterior sampling from the current diffusion prior given observations with denoising score matching on those samples (EM), and use a moment-matching posterior sampler that estimates the posterior covariance via Tweedie's covariance formula and conjugate gradient, yielding proper denoisers unlike prior ambient-diffusion methods.
Key results
- On corrupted CIFAR-10, DiEM at 75% corruption performs similarly to AmbientDiffusion at only 20% corruption, while heuristic covariance choices yield poor sample quality.
- On FFHQ inverse problems, MMPS with 5 solver iterations achieves the best LPIPS/PSNR/SSIM in most tasks, e.g. 26.70 PSNR on motion deblur at 10 steps versus 25.02 for DPS at 1000 steps on super resolution-type comparisons.
- MMPS is sensitive to solver iterations: 1 to 3 CG iterations give large gains while 3 to 5 gives diminishing returns.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MMPS instead of prior posterior-sampling baselines | Pretrained diffusion priors on FFHQ inverse problems · Box inpainting, random inpainting, motion deblur, and 4× super resolution | Image reconstruction quality | Helped |
| Increase MMPS solver iterations from 1 to 5 (ablation) | MMPS with a pretrained diffusion prior · Motion deblur | PSNR: 22.12 -> 26.70 | Helped |
| Increase MMPS solver iterations from 1 to 5 (ablation) | MMPS with a pretrained diffusion prior · Box inpainting | LPIPS: 0.27 -> 0.23 | Helped |
Surprising findings
- Heuristic covariance approximations (e.g. Sigma_t) cause EM collapse on accelerated MRI after a few iterations, while Tweedie covariance remains stable.
Limitations
- Restricted to linear Gaussian forward processes; non-linear or black-box forward models are left to future work.
- Iterative EM with posterior sampling is expensive compared with prior ambient methods.
- Empirical Bayes is ill-posed, so the learned prior may differ from the true prior even when consistent with observations.
Experimental setup
Datasets: CIFAR-10 fastMRI FFHQ
Benchmarks: CIFAR-10 corrupted observations (IS, FID) accelerated MRI qualitative samples FFHQ linear inverse problems (LPIPS, PSNR, SSIM) Sinkhorn divergence on low-dimensional manifold
Metrics: Inception score FID LPIPS PSNR SSIM Sinkhorn divergence
Baselines: AmbientDiffusion GSURE diffusion (Kawar2024GSURE) DPS DiffPIR ΠGDM TMPD heuristic covariance posterior sampling
Builds on: Monte Carlo EM Ambient Diffusion DPS Score-based SDEs Tweedie's formula
Closest papers by meaning
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.92)
- Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors (NeurIPS 2024 · similarity 0.91)
- An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations (NeurIPS 2024 · similarity 0.91)
- Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.91)
- Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.89)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.