Research map / Image generation / Super-resolution
Restabilizing Diffusion Models with Predictive Noise Fusion Strategy for Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PNFS, a plug-and-play strategy that predicts pixel-wise restoration errors from the initial noise and fuses two Gaussian noises via probabilistic mask sampling to stabilize and improve diffusion-based super-resolution.
Problem
Diffusion-based super-resolution is unstable: results vary significantly with the random initial Gaussian noise, and existing remedies need multiple runs or retraining.
Approach
A lightweight Discrepancy Prediction Module predicts per-pixel quality of a noise's restoration; a Probabilistic Fusion Module then samples a binary mask from softmax-weighted inverse discrepancies to merge two noises into a fused initial noise that stays close to N(0, I).
Key results
- On Set5 4x, PNFS raises LDM PSNR from 26.67 to 27.06 dB and cuts L-STD from 0.031 to 0.024; with SD-Upscaler, PSNR rises by 1.37 dB on Set5.
- On RealSR 4x, PNFS on StableSR reaches 26.39 dB PSNR, LPIPS 0.292 and FID 120.1, versus 24.69 dB, 0.309 and 127.2 for StableSR.
- PNFS adds only about 0.03 s inference time and a 13.69 MB parameter size, about 0.1% of the backbone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PNFS to LDM | LDM · Set5, Set14, BSD100, Urban100, Manga109 (4×) | PSNR: 26.67 -> 27.06; 24.80 -> 25.20; 24.56 -> 24.91; 22.76 -> 23.24; 25.28 -> 25.72 | Helped |
| Add PNFS to SD-Upscaler | SD-Upscaler · Set5, Set14, BSD100, Urban100, Manga109 (4×) | PSNR: 25.05 -> 26.42; 24.60 -> 25.87; 24.03 -> 24.61; 23.41 -> 23.60; 25.38 -> 26.27 | Helped |
| Add PNFS to StableSR | StableSR · Set5, Set14, BSD100, Urban100, Manga109 (4×) | PSNR: 24.28 -> 24.81; 24.22 -> 24.54; 24.10 -> 24.35; 22.74 -> 22.93; 24.75 -> 25.27 | Helped |
| Add PNFS to StableSR | StableSR · RealSR 2×, 3×, 4× | PSNR: 27.45 -> 27.69; 27.16 -> 27.37; 24.69 -> 26.39 | Helped |
| Use 2 candidate noises rather than 1 (ablation) | PNFS with LDM · Set5 | PSNR: 26.67 -> 27.06 | Helped |
| Increase candidate noise count from 2 to 5 (ablation) | PNFS with LDM · Set5 | PSNR: 27.06 -> 27.26 | Helped |
| Increase candidate noise count from 5 to 10 (ablation) | PNFS with LDM · Set5 | PSNR: 27.26 -> 27.69 | Helped |
| Increase candidate noise count from 10 to 50 (ablation) | PNFS with LDM · Set5 | PSNR: 27.69 -> 27.60 | Hurt |
| Increase candidate noise count from 10 to 100 (ablation) | PNFS with LDM · Set5 | PSNR: 27.69 -> 27.65 | Hurt |
| Conduct PNFS at reverse step T rather than no PNFS (ablation) | PNFS with LDM · Set14 | PSNR: 26.67 -> 27.06 | Helped |
| Conduct PNFS at reverse step T-5 rather than T (ablation) | PNFS with LDM · Set14 | PSNR: 27.06 -> 27.14 | Helped |
| Conduct PNFS at reverse step T-10 rather than T (ablation) | PNFS with LDM · Set14 | PSNR: 27.06 -> 27.55 | Helped |
Surprising findings
- Fusing noise with a random uniform mask gives results similar to other random noises, far below the predictive mask, so the learned prediction (not just mixing) drives the gain.
Limitations
- Pseudo labels for DPM training depend on running the full diffusion sampling for each noise, which is costly to generate.
- Performance gains on some datasets are modest, and the paper reports that the fused noise deviates from N(0, I) when more candidate noises are used.
Experimental setup
Datasets: DIV2K Flickr2K Set5 Set14 BSD100 Urban100 Manga109 RealSR
Benchmarks: Set5 Set14 BSD100 Urban100 Manga109 RealSR
Metrics: PSNR SSIM LPIPS FID L-STD
Baselines: LDM SD-Upscaler StableSR BCs CCSR SwinIR
Builds on: DDIM LPIPS StableSR Latent Diffusion Models
Closest papers by meaning
- Arbitrary-steps Image Super-resolution via Diffusion Inversion (CVPR 2025 · similarity 0.92)
- Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution (ICLR 2024 · similarity 0.89)
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution (NeurIPS 2024 · similarity 0.89)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.89)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.89)
- Efficient Conditional Diffusion Model with Probability Flow Sampling for Image Super-resolution (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.