Research map / Image generation / Image restoration
Blind Image Restoration via Fast Diffusion Inversion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHRED, a blind image restoration method that optimizes only the initial DDIM noise and degradation parameters at test time, using fast large-step DDIM inversion to keep restorations on the data manifold.
Problem
Blind image restoration with unknown degradation parameters, where existing diffusion-based IR methods alter the reverse sampling and can produce outputs off the natural image manifold.
Approach
Keep the DDIM reverse sampling unaltered and cast restoration as optimization over the initial noise (plus degradation parameters), with large-step DDIM reverse sampling (step size 100) to make the unrolled inversion computationally tractable.
Key results
- On CelebA 8x super-resolution, SHRED reaches 22.75 PSNR / 0.306 LPIPS versus 21.82 / 0.345 for BlindDPS.
- On ImageNet 4x super-resolution, SHRED reaches 22.15 PSNR / 0.354 LPIPS, above BlindDPS at 21.36 / 0.374.
- Using delta t=100 reduces per-image runtime to 234 s on CelebA deblurring versus 5700 s for naive full unrolled inversion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHRED instead of Fast Diffusion EM for motion deblurring | CelebA validation · Motion Deblur | PSNR: 23.18 -> 23.76 | Helped |
| Use SHRED instead of BlindDPS for motion deblurring | CelebA validation · Motion Deblur | LPIPS: 0.281 -> 0.263 | Helped |
| Use SHRED instead of Fast Diffusion EM for Gaussian deblurring | CelebA validation · Gaussian Deblur | PSNR: 24.52 -> 24.67 | Helped |
| Use SHRED instead of BlindDPS for Gaussian deblurring | ImageNet validation · Gaussian Deblur | PSNR: 24.28 -> 23.76 | Hurt |
| Use SHRED instead of BlindDPS for Gaussian deblurring | ImageNet validation · Gaussian Deblur | LPIPS: 0.296 -> 0.321 | Hurt |
| Use SHRED instead of BlindDPS for super-resolution | CelebA validation · 8× SR | PSNR: 21.82 -> 22.75 | Helped |
| Use SHRED instead of BlindDPS for super-resolution | ImageNet validation · 4× SR | LPIPS: 0.374 -> 0.354 | Helped |
| Set diffusion step size δt to 50 instead of 100 (ablation) | SHRED · CelebA, Denoising | PSNR: 28.67 -> 28.74 | No clear effect |
| Set diffusion step size δt to 50 instead of 100 (ablation) | SHRED · CelebA, Denoising | Time[s]: 234 -> 412 | Hurt |
| Set diffusion step size δt to 200 instead of 100 (ablation) | SHRED · CelebA, Denoising | PSNR: 28.67 -> 28.45 | No clear effect |
| Set diffusion step size δt to 200 instead of 100 (ablation) | SHRED · CelebA, Denoising | Time[s]: 234 -> 110 | Helped |
Surprising findings
- On ImageNet Gaussian deblurring, BlindDPS obtains higher PSNR (24.28) than SHRED (23.76), so SHRED is not best on every task.
Limitations
- Evaluated only on degradations with explicit parametrized forms; not tested on deraining or dehazing.
- Per-image test-time optimization is iterative and slower than feed-forward methods.
- Inherits biases of the pre-trained diffusion model.
Experimental setup
Datasets: ImageNet validation CelebA validation
Benchmarks: CelebA ImageNet
Metrics: PSNR LPIPS runtime
Baselines: BlindDPS DIP Fast Diffusion EM GibbsDDRM DGPGAN GDP DPS
Builds on: DDIM DPS BlindDPS GDP DDRM
Closest papers by meaning
- A Diffusion Model with State Estimation for Degradation-Blind Inverse Imaging (AAAI 2024 · similarity 0.90)
- Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction (AAAI 2024 · similarity 0.90)
- Taming Generative Diffusion Prior for Universal Blind Image Restoration (NeurIPS 2024 · similarity 0.90)
- A Modular Conditional Diffusion Framework for Image Reconstruction (NeurIPS 2024 · similarity 0.89)
- Residual Denoising Diffusion Models (CVPR 2024 · similarity 0.89)
- Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.