Research map / Image generation / Image restoration
Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reframes noisy inverse problems as a two-variable constrained optimization over x0 and an auxiliary noisy sample at an equivalent diffusion step, solved by projected gradient descent with gradient truncation through the mu-predictor (ProjDiff).
Problem
Solving noisy linear and nonlinear inverse problems with pretrained diffusion priors within a MAP/optimization framework, where existing methods ignore the denoising capability of diffusion models when handling observation noise.
Approach
Introduce an auxiliary variable x_{t_a} representing a noisy sample at the diffusion step whose noise level matches the observation noise, derive a two-variable ELBO on the joint log-prior, and optimize with projected gradient descent while truncating gradients through the denoiser.
Key results
- On ImageNet noisy (sigma=0.05) super-resolution, inpainting and Gaussian deblurring with 100 NFEs, ProjDiff reports the best PSNR on inpainting (31.09 dB vs 30.00 for DDNM+) and deblurring (27.91 dB vs 27.90 for DDNM+).
- On FFHQ noise-free phase retrieval with 1000 NFEs, ProjDiff reaches 33.39 PSNR and 20.42 LPIPS vs 13.19 PSNR for DPS.
- On source separation (SI-SDRi average), ProjDiff reaches 19.23 vs 17.73 for Demucs+Gibbs, the previous SOTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ProjDiff instead of DPS for noisy ImageNet super-resolution | ImageNet 256×256 diffusion model · ImageNet noisy super-resolution, σ=0.05 | PSNR: 24.44 -> 25.73 | Helped |
| Use ProjDiff instead of DDRM for noisy ImageNet inpainting | ImageNet 256×256 diffusion model · ImageNet noisy inpainting, σ=0.05 | LPIPS ×100: 17.11 -> 13.65 | Helped |
| Use ProjDiff instead of DDNM+ for noisy ImageNet Gaussian deblurring | ImageNet 256×256 diffusion model · ImageNet noisy Gaussian deblurring, σ=0.05 | FID: 46.54 -> 32.27 | Helped |
| Use ProjDiff instead of DPS for phase retrieval | FFHQ phase retrieval · FFHQ phase retrieval, σ=0 | PSNR: 13.19 -> 33.39 | Helped |
| Use ProjDiff instead of DPS for noisy phase retrieval | FFHQ phase retrieval · FFHQ phase retrieval, σ=0.1 | PSNR: 12.58 -> 23.84 | Helped |
| Use ProjDiff instead of Demucs+Gibbs | ISDM source separation · SLACK2100 source separation | SI-SDR_i average: 17.73 -> 19.23 | Helped |
| Apply Restricted Encoding | ProjDiff · SLACK2100 partial generation | sub-FAD | No clear effect |
| Use decreasing time steps from T to 1 | ProjDiff · Inverse problems | performance | Helped |
| Repeat the same time step multiple times | ProjDiff · Inverse problems | performance | Helped |
| Set DDIM variance to zero for t≤t_a | ProjDiff · Inverse problems | performance | Helped |
Surprising findings
- A diffusion-based separation method without task-specific training surpasses the Demucs-based previous SOTA on source separation.
Limitations
- Potential difficulty handling noisy observations with highly intricate nonlinear functions.
- Reliance on Gaussian observation noise limits applicability to Poisson or multiplicative noise.
- Step sizes require manual tuning.
Experimental setup
Datasets: ImageNet CelebA FFHQ SLAKH2100
Benchmarks: ImageNet 1k subset (super-resolution, inpainting, Gaussian deblurring) FFHQ phase retrieval SLAKH2100 source separation SLAKH2100 partial generation
Metrics: PSNR SSIM LPIPS FID SI-SDR_i sub-FAD
Baselines: DDRM DDNM+ DPS RED-diff ER HIO OSS ISDM-Gaussian ISDM-Dirac Demucs+Gibbs MSDM
Builds on: DPS RED-diff DDNM DDRM DDIM
Closest papers by meaning
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.92)
- Diffusion State-Guided Projected Gradient for Inverse Problems (ICLR 2025 · similarity 0.91)
- Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing (CVPR 2025 · similarity 0.91)
- Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance (ICML 2024 · similarity 0.91)
- A Diffusion Model with State Estimation for Degradation-Blind Inverse Imaging (AAAI 2024 · similarity 0.90)
- Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.