Research map / Image generation / Image restoration
A Modular Conditional Diffusion Framework for Image Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DP-IR, a modular conditional diffusion framework that combines frozen pre-trained IR networks and a denoiser with a small trained fusion module, plus a one-step accelerated sampling strategy that skips early reverse steps.
Problem
Diffusion probabilistic models for blind image restoration are task-specific and expensive to train, and require hundreds of neural function evaluations for sampling.
Approach
Decompose the conditional denoiser into a frozen pre-trained IR network's estimate of E[x0|y], a frozen unconditional denoiser's estimate of E[x0|x_t], and a small task-specific fusion network; skip the first T-tau reverse steps by using the IR estimate directly, so only tau steps need full evaluation.
Key results
- On GoPro deblurring, DP-IR reaches LPIPS 0.053 with 6 NFEs, versus 0.058 for InDI at 10 NFEs, and PSNR 33.72 vs 31.49 for InDI.
- On DIV2K 4x SISR, DP-IR reaches PSNR 28.12 and TOPIQ_delta 0.002 with 51 NFEs, the best TOPIQ among all compared methods.
- Fusion module has 0.7M parameters versus 30-31M for U-Net and L-DWT fusion baselines in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DP-IR instead of FFTFormer | Dynamic scene deblurring · GoPro | LPIPS: 0.071 -> 0.053 | Helped |
| Use DP-IR instead of FFTFormer | Dynamic scene deblurring · HIDE | LPIPS: 0.096 -> 0.087 | Helped |
| Use DP-IR instead of FFTFormer | Dynamic scene deblurring · GoPro | PSNR: 34.21 -> 33.72 | Hurt |
| Use DP-IR instead of HAT | 4× SISR · DIV2K validation | PSNR: 29.75 -> 28.12 | Hurt |
| Use DP-IR instead of ESRGAN | 4× SISR · DIV2K validation | LPIPS: 0.115 -> 0.140 | Hurt |
| Use DP-IR instead of SRDiff | 4× SISR · DIV2K validation | TOPIQ_delta: 0.008 -> 0.002 | Helped |
| Replace UDP with MIRNet-S denoiser while keeping RRDB (ablation) | DP-IR · DIV2K validation, 4× SISR | PSNR: 27.93 -> 28.12 | Helped |
| Replace RRDB with SwinIR IR network while keeping MIRNet-S (ablation) | DP-IR · DIV2K validation, 4× SISR | LPIPS: 0.150 -> 0.140 | Helped |
| Replace TDWA fusion with proposed fusion (ablation) | DP-IR using UDP and RRDB · DIV2K validation, 4× SISR | LPIPS: 0.266 -> 0.149 | Helped |
| Replace L-DWT fusion with proposed fusion (ablation) | DP-IR using UDP and RRDB · DIV2K validation, 4× SISR | PSNR: 29.27 -> 27.93 | Hurt |
| Increase tau from 50 to 250 (ablation) | DP-IR using UDP and RRDB · DIV2K validation, 4× SISR | PSNR: 28.77 -> 27.93 | Hurt |
| Increase tau from 50 to 250 (ablation) | DP-IR using UDP and RRDB · DIV2K validation, 4× SISR | LPIPS: 0.170 -> 0.149 | Hurt |
Surprising findings
- Skipping the first T-tau reverse steps by using the IR estimate directly reportedly leaves reconstruction unchanged, with tau as small as 5 of 1000 steps for deblurring.
Limitations
- Performance depends on the quality of the employed denoising and IR modules; inapplicable where no pre-trained IR network exists.
- Optimal tau is task-dependent and selected experimentally per dataset; no reliable per-sample selection method.
- Fusion architecture is a proof of concept without extensive search for the optimal design.
Experimental setup
Datasets: DF2K DIV2K Flickr2K ZurichRAW2RGB GoPro
Benchmarks: SyntheticBurst GoPro HIDE DIV2K validation
Metrics: PSNR SSIM LPIPS TOPIQ_delta NFE
Baselines: BSRT-Small BIPNet EBSR DBSR NAFNet FFTFormer DvSR InDI icDPM SRDiff LDM IDM
Builds on: DDPM DDIM denoising score matching 3CA
Closest papers by meaning
- Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual (CVPR 2025 · similarity 0.90)
- Blind Image Restoration via Fast Diffusion Inversion (NeurIPS 2024 · similarity 0.89)
- Learning Diffusion Texture Priors for Image Restoration (CVPR 2024 · similarity 0.89)
- Taming Generative Diffusion Prior for Universal Blind Image Restoration (NeurIPS 2024 · similarity 0.89)
- Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction (NeurIPS 2024 · similarity 0.89)
- On Exact Inversion of DPM-Solvers (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.