Research map / Image generation / Image restoration
A Diffusion Model with State Estimation for Degradation-Blind Inverse Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a degradation-blind diffusion framework for inverse imaging that learns a State Estimator producing a pixel-wise mask to blend noised measurements with the diffusion intermediate state, fine-tuning pretrained latent diffusion models on 1k-2k pairs.
Problem
Restoring clean images from degraded measurements (inpainting, deblurring, JPEG restoration) when the degradation operator is unknown or non-linear, using pretrained diffusion priors without retraining conditional models from scratch.
Approach
A learnable U-Net State Estimator outputs a per-pixel weighting mask that combines the noised measurement with the diffusion intermediate state at each timestep; the diffusion model is fine-tuned with truncated backpropagation through time using an MSE+LPIPS loss on predicted x0 decoded by the LDM decoder.
Key results
- On FFHQ Gaussian deblurring, the method reaches PSNR 27.98 and LPIPS 0.0939 versus DPS 25.84 and 0.1279.
- On inpainting, the method achieves FID 12.923 on FFHQ, close to RePaint's 12.504 but with latency 5.32 s versus RePaint's 272.45 s.
- On JPEG QF=5 FFHQ, the method reaches LPIPS 0.1237 versus DDRM's 0.2000, though its PSNR (25.43) is below DDRM's 26.25.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the state estimator with measurement concatenation (ablation) | State-estimator diffusion model · Inpainting, FFHQ 256×256 | LPIPS: 0.0681 -> 0.0530 | Helped |
| Replace the state estimator with measurement concatenation (ablation) | State-estimator diffusion model · Gaussian deblurring, FFHQ 256×256 | LPIPS: 0.0848 -> 0.0939 | No clear effect |
| Replace the state estimator with measurement concatenation (ablation) | State-estimator diffusion model · Motion deblurring, FFHQ 256×256 | LPIPS: 0.1394 -> 0.1112 | Helped |
| Replace the state estimator with measurement concatenation (ablation) | State-estimator diffusion model · JPEG restoration QF=5, FFHQ 256×256 | LPIPS: 0.1425 -> 0.1237 | Helped |
| Replace the state estimator with measurement concatenation (ablation) | State-estimator diffusion model · JPEG restoration QF=10, FFHQ 256×256 | LPIPS: 0.1255 -> 0.1068 | Helped |
| Use the proposed method instead of RePaint | Inpainting · FFHQ 256×256 | latency (s): 272.45 -> 5.32 | Helped |
| Use the proposed method instead of MPRNet | Gaussian deblurring · FFHQ 256×256 | LPIPS: 0.0976 -> 0.0939 | Helped |
| Use the proposed method instead of DDRM | JPEG restoration · FFHQ 256×256, QF=10 | LPIPS: 0.1341 -> 0.1068 | Helped |
Surprising findings
- The method has worse PSNR than DDRM on JPEG QF=5 FFHQ despite better LPIPS, and worse FID than RePaint on inpainting, indicating mixed rather than uniform gains.
Limitations
- Solution space is limited by the generative ability of the pretrained diffusion model.
- Evaluated only on FFHQ and LSUN-Bedroom at 256x256 with three tasks.
- Mixed results versus baselines on some metrics (e.g., PSNR for JPEG QF=5, FID for inpainting).
Experimental setup
Datasets: FFHQ LSUN-Bedroom
Benchmarks: FFHQ 256x256 LSUN-Bedroom 256x256
Metrics: PSNR LPIPS FID user study votes latency
Baselines: MAT PUT DDRM RePaint DPS DeblurGANV2 MPRNet BlindDPS GDP QGAC DDRM-JPEG
Builds on: latent diffusion models RePaint DDIM Palette TBPTT
Closest papers by meaning
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.92)
- Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems (NeurIPS 2024 · similarity 0.90)
- Blind Image Restoration via Fast Diffusion Inversion (NeurIPS 2024 · similarity 0.90)
- Improving Diffusion-Based Image Restoration with Error Contraction and Error Correction (AAAI 2024 · similarity 0.90)
- Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Data (ICLR 2025 · similarity 0.90)
- Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.