Research map / Image generation / Image restoration
Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CM4IR, a zero-shot image restoration scheme using pretrained consistency models with 4 NFEs, combining better initialization, back-projection guidance, and a novel split noise injection.
Problem
Zero-shot diffusion-based image restoration requires many neural function evaluations (tens to a thousand), and existing guided consistency model methods need tens of NFEs or task-specific fine-tuning that is not zero-shot.
Approach
Combine median/pseudoinverse initialization, back-projection guidance, and a noise injection that decouples the denoiser noise level from the injection level ((1+δ)τn) and splits injected noise between stochastic noise and an estimated noise term (x0|τn − xτn)/τn, motivated as a noisy Polyak momentum; this enables reliable restoration with only 4 NFEs.
Key results
- CM4IR with 4 NFEs reaches 25.60 dB PSNR on SRx4 (sigma 0.05, LSUN Bedroom) versus 23.62 dB for CM with 40 NFEs and 25.08 dB for DDRM with 20 NFEs.
- On 80% inpainting (sigma 0.05, LSUN Bedroom), CM4IR achieves 25.28 dB vs 22.38 dB for DiffPIR (20 NFEs).
- Applying the sign-flipped noise injection to DDRM with 4 NFEs raises SRx4 PSNR above its 20-NFE original implementation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CM4IR instead of the strongest zero-shot DM baseline | Image restoration with pretrained consistency models · LSUN Bedroom val, SRx4 σy=0.025 | PSNR [dB]: 25.67 -> 26.14 | Helped |
| Use CM4IR instead of the strongest zero-shot DM baseline | Image restoration with pretrained consistency models · LSUN Bedroom val, Gaussian deblurring σy=0.05 | PSNR [dB]: 27.35 -> 27.42 | Mixed |
| Use CM4IR instead of the strongest zero-shot DM baseline | Image restoration with pretrained consistency models · LSUN Bedroom val, inpainting (80%) σy=0.05 | PSNR [dB]: 19.09 -> 25.28 | Helped |
| Use CM4IR instead of the strongest zero-shot DM baseline | Image restoration with pretrained consistency models · LSUN Cat val, SRx4 σy=0.025 | PSNR [dB]: 26.93 -> 27.18 | Helped |
| Use CM4IR instead of the strongest zero-shot DM baseline | Image restoration with pretrained consistency models · LSUN Cat val, Gaussian deblurring σy=0.025 | PSNR [dB]: 29.84 -> 29.76 | Mixed |
| Use CM4IR instead of the strongest zero-shot DM baseline | Image restoration with pretrained consistency models · LSUN Cat val, inpainting (80%) σy=0.025 | PSNR [dB]: 18.51 -> 25.89 | Helped |
| Use the proposed estimated-noise injection ẑ− instead of DDIM noise ẑ (ablation) | CM4IR, SRx4 · LSUN Bedroom val, σy=0.025 | PSNR [dB]: 24.64 -> 25.94 | Helped |
| Use the proposed estimated-noise injection ẑ− instead of DDIM noise ẑ (ablation) | CM4IR, SRx4 · LSUN Bedroom val, σy=0.05 | PSNR [dB]: 20.95 -> 25.51 | Helped |
| Set η=1, removing the estimated-noise contribution (ablation) | CM4IR, SRx4 · LSUN Bedroom val, σy=0.025 | PSNR [dB]: 24.49 -> 25.94 | Hurt |
| Replace ẑ− with Polyak direction βv (ablation) | CM4IR, SRx4 · LSUN Bedroom val, σy=0.025 | PSNR [dB]: 23.37 -> 25.94 | Hurt |
| Increase denoiser noise level using δ>0 (ablation) | CM4IR, SRx4 · LSUN Bedroom val, σy=0.025 | PSNR [dB]: 25.94 -> 26.14 | Helped |
| Reduce guided DM sampling from 20 to 4 NFEs and use auto-calculated time points (ablation) | DDRM · LSUN Bedroom val, SRx4 σy=0.025 | PSNR [dB]: 25.67 -> 24.16 | Hurt |
Surprising findings
- Using the DDIM-style noise sign (pulling toward the current iterate) or plain Polyak momentum deteriorates few-step restoration relative to the proposed noise injection.
- The sign-flipped noise injection lets DDRM and DiffPIR with 4 NFEs match or exceed their 20-NFE performance in some SR settings.
Limitations
- Results are limited to three tasks (SR, deblurring, inpainting) on LSUN Bedroom/Cat at 256x256, with additional ImageNet results only in supplementary material.
- Requires tuning of time points and hyperparameters (gamma, alpha_N, delta, eta) per task.
- CM4IR does not outperform CoSIGN on its own trained SRx4 setting (sigma 0.025).
Experimental setup
Datasets: LSUN Bedroom LSUN Cat
Benchmarks: LSUN Bedroom validation (300 images) LSUN Cat validation (300 images)
Metrics: PSNR LPIPS
Baselines: CM (40 NFEs) CoSIGN DDRM DiffPIR DPS
Builds on: Consistency Models DDRM DDIM Polyak acceleration DiffPIR
Closest papers by meaning
- Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance (CVPR 2024 · similarity 0.88)
- Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual (CVPR 2025 · similarity 0.87)
- Zero-shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model (AAAI 2025 · similarity 0.86)
- Masked Pre-training Enables Universal Zero-shot Denoiser (NeurIPS 2024 · similarity 0.86)
- Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework (NeurIPS 2025 · similarity 0.86)
- Boosting Image Restoration via Priors from Pre-trained Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.