Research map / Image generation / Image restoration
Residual Denoising Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Residual Denoising Diffusion Models (RDDM), a dual diffusion process that splits the forward process into residual diffusion and noise diffusion to unify image generation and restoration with a generic UNet trained with L1 loss.
Problem
Denoising diffusion models for image restoration start from pure noise and implicitly use the degraded image as a condition, making the forward process uninformative about the degraded input and the reverse process from noise unnecessary.
Approach
Redefine the forward step so that the terminal state is I_T = I_in + noise, decomposing diffusion into a directional residual diffusion (target to degraded input) and an independent noise diffusion with two separate coefficient schedules; the reverse process predicts residuals and/or noise, with a partially path-independent sampler and conversion from DDPM/DDIM schedules.
Key results
- Converting a pre-trained DDIM to RDDM keeps CelebA generation FID nearly unchanged (e.g., 40.45 vs 40.41 at 10 steps).
- On ISTD shadow removal, RDDM reaches PSNR 30.91 overall, above DMTN (30.42) and BMNet (30.28) with 5 sampling steps.
- On RainDrop deraining, RDDM achieves PSNR 32.51 and SSIM 0.9563, exceeding RainDiff128 (32.43 PSNR).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| convert DDIM coefficients to RDDM coefficients | DDIM image generation on CelebA 256×256 · CelebA 256×256 | FID: 23.66 -> 24.92 | No clear effect |
| use RDDM for shadow removal | ISTD shadow removal · ISTD | PSNR (ALL): 30.42 -> 30.91 | Helped |
| use RDDM for low-light enhancement | LOL low-light enhancement · LOL | PSNR: 25.19 -> 25.392 | Helped |
| use RDDM for low-light enhancement | LOL low-light enhancement · LOL | LPIPS: 0.11 -> 0.116 | Hurt |
| use RDDM for deraining | RainDrop deraining · RainDrop | PSNR: 32.43 -> 32.51 | Helped |
| predict noise only instead of residuals only (ablation) | RDDM image generation on CelebA 64×64 · CelebA 64×64 | FID: 31.47 -> 23.25 | Helped |
| predict noise only instead of residuals only (ablation) | RDDM shadow removal on ISTD · ISTD | PSNR: 30.72 -> 11.34 | Hurt |
| predict residuals and noise instead of residuals only (ablation) | RDDM image generation on CelebA 64×64 · CelebA 64×64 | FID: 31.47 -> 28.90 | Hurt |
| predict residuals and noise instead of residuals only (ablation) | RDDM shadow removal on ISTD · ISTD | PSNR: 30.72 -> 30.91 | Helped |
| use linearly decreasing αt and linearly increasing βt² schedules (ablation) | RDDM generation on CelebA 64×64 · CelebA 64×64 | FID: 28.394 -> 23.25 | Helped |
Surprising findings
- Converting DDIM schedules to RDDM preserves generation quality without retraining, and readjusting the residual schedule alone barely affects generated images when the noise schedule is fixed.
- Residual-only prediction (SM-Res) gives poor FID for generation, while noise-only prediction (SM-N) gives poor restoration PSNR (e.g., 11.34 on ISTD), suggesting a certainty-diversity trade-off.
Limitations
- Generation FID is worse than the original DDPM/DDIM setup when trained with the authors' own network and training details.
- Path independence is only partial; excessive schedule perturbations cause visual inconsistencies.
- Evaluation is limited to a few restoration tasks and small-resolution datasets, with no large-scale generation benchmarks.
Experimental setup
Datasets: CelebA ISTD LOL RainDrop
Benchmarks: CelebA 256x256 ISTD LOL RainDrop
Metrics: FID IS MAE PSNR SSIM LPIPS
Baselines: DSC FusionNet BMNet DMTN KinD++ SNR-Aware LLFlow LLFormer AttnGAN DuRN IDT RainDiff
Builds on: DDPM DDIM Rectified Flow I2SB InDI
Closest papers by meaning
- Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise (NeurIPS 2024 · similarity 0.92)
- Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction (AAAI 2024 · similarity 0.91)
- Directly Denoising Diffusion Models (ICML 2024 · similarity 0.91)
- Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual (CVPR 2025 · similarity 0.90)
- Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration (ICLR 2025 · similarity 0.90)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.