Research map / Image generation / Image restoration
Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RDMD, a zero-shot image restoration framework that uses one pre-trained diffusion model to build both a deterministic (RED-style) regularizer and a stochastic (DiffPIR-style) sampling regularizer, balanced by a single hyperparameter τ.
Problem
Zero-shot image restoration with plug-and-play and diffusion priors faces a trade-off: deterministic denoiser-based regression gives high fidelity but artifacts, while stochastic diffusion sampling gives perceptual quality but loses fidelity; prior combinations need extra networks.
Approach
Use a single pre-trained diffusion model's noise predictor in two roles within an HQS-style iteration: a stochastic reverse-diffusion step for the x sub-problem and a deterministic RED-style gradient step for the z sub-problem, with τ controlling the distortion-perception tradeoff.
Key results
- On FFHQ 8x super-resolution (sigma 0.05), RDMD reaches 23.31 dB PSNR vs 21.08 dB for DiffPIR, a gain of about 2.2 dB.
- On FFHQ 2x super-resolution, RDMD reaches 31.03 dB PSNR and 0.1807 LPIPS, the best among compared methods.
- On ImageNet 4x super-resolution, RDMD reaches 23.28 dB PSNR and 0.4000 LPIPS, outperforming the baselines on most metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reconcile deterministic regression with stochastic sampling | RDMD · FFHQ 256×256, Gaussian deblur | PSNR: 27.84 -> 28.29 | Helped |
| Reconcile deterministic regression with stochastic sampling | RDMD · FFHQ 256×256, super-resolution (8×) | PSNR: 22.22 -> 23.31 | Helped |
| Reconcile deterministic regression with stochastic sampling | RDMD · ImageNet 256×256, super-resolution (8×) | PSNR: 20.42 -> 20.83 | Helped |
| Reconcile deterministic regression with stochastic sampling | RDMD · ImageNet 256×256, motion deblur | PSNR: 23.34 -> 22.77 | Hurt |
| Reconcile deterministic regression with stochastic sampling | RDMD · FFHQ 256×256, super-resolution (8×) | LPIPS: 0.4883 -> 0.3614 | Helped |
| Use stochastic sampling instead of deterministic regression (ablation) | RDMD · FFHQ 256×256, super-resolution (8×) | PSNR: 22.22 -> 21.08 | Hurt |
| Use stochastic sampling instead of deterministic regression (ablation) | RDMD · FFHQ 256×256, super-resolution (8×) | LPIPS: 0.4883 -> 0.3666 | Helped |
| Increase iteration count to T = 100 (ablation) | RDMD · FFHQ 256×256, super-resolution (4×) | PSNR | Helped |
| Increase iteration count beyond T = 100 (ablation) | RDMD · FFHQ 256×256, super-resolution (4×) | PSNR | Hurt |
| Increase iteration count (ablation) | RDMD · FFHQ 256×256, super-resolution (4×) | LPIPS | Helped |
| Set λ too low (λ = 10) (ablation) | RDMD · FFHQ 256×256, super-resolution (4×) | PSNR | Hurt |
| Increase λ from 20 to 50 (ablation) | RDMD · FFHQ 256×256, super-resolution (4×) | PSNR | Hurt |
Surprising findings
- Replacing the discriminative denoiser with a second network in the dual-prior setting (DRUNet or SwinIR) did not outperform the single diffusion model version, which is consistent with the single-model design being sufficient.
Limitations
- Single noise level sigma_n=0.05 and a fixed set of degradations evaluated; generalization to other noise levels not shown in the main text.
- Hyperparameters lambda and tau require tuning per task to balance distortion and perception.
Experimental setup
Datasets: FFHQ 256x256 ImageNet 256x256
Benchmarks: FFHQ 256x256 ImageNet 256x256
Metrics: PSNR SSIM LPIPS
Baselines: MCG DPS Resample BIRD RED DiffPIR
Builds on: plug-and-play regularization by denoising DDIM DiffPIR DPS
Closest papers by meaning
- Zero-shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model (AAAI 2025 · similarity 0.91)
- Residual Denoising Diffusion Models (CVPR 2024 · similarity 0.90)
- Plug-and-Play image restoration with Stochastic deNOising REgularization (ICML 2024 · similarity 0.90)
- A Modular Conditional Diffusion Framework for Image Reconstruction (NeurIPS 2024 · similarity 0.90)
- Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction (AAAI 2024 · similarity 0.90)
- Blind Image Restoration via Fast Diffusion Inversion (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.