Research map / Image generation / Image restoration
Learning Image Priors Through Patch-Based Diffusion Models for Solving Inverse Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a diffusion denoiser only on randomly positioned image patches with positional encodings, and composes whole-image scores from patch scores for plug-in use in diffusion inverse solvers (PaDIS).
Problem
Diffusion-based inverse problem solvers need large training sets and heavy compute, and become infeasible for high-resolution and 3D images because the whole image must be fed to the network.
Approach
Learn the score of patches with a network conditioned on positional arrays, then approximate the whole-image score by randomly choosing zero-padded grid partitions each sampling iteration, so the whole image is never input to the network and boundary artifacts are avoided.
Key results
- PaDIS reaches 33.57 dB PSNR / 0.854 SSIM on 20-view CT, above the whole-image diffusion baseline at 32.84 dB / 0.835.
- With 144 training images, patch-based CT reconstruction gets 32.28 dB PSNR versus 29.12 dB for the whole-image model.
- On 60-view 512x512 CT, PaDIS reaches 36.93 dB PSNR versus 29.48 dB for ADMM-TV.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use patch-based PaDIS instead of whole-image diffusion | CT reconstruction · CT, 8 Views | PSNR: 25.74 -> 29.48 | Helped |
| Use patch-based PaDIS instead of whole-image diffusion | Deblurring · Deblurring | SSIM: 0.853 -> 0.870 | Helped |
| Use patch-based PaDIS instead of whole-image diffusion | Superresolution · Superresolution | PSNR: 29.17 -> 29.47 | Helped |
| Increase patch size from 8 to 56 (ablation) | PaDIS · CT, 20 Views | PSNR: 32.57 -> 33.57 | Helped |
| Use patch size 56 rather than patch size 16 (ablation) | PaDIS · CT, 20 Views | PSNR: 32.57 -> 33.57 | Helped |
| Use patch size 56 rather than patch size 32 (ablation) | PaDIS · CT, 20 Views | PSNR: 32.72 -> 33.57 | Helped |
| Use patch size 56 rather than patch size 96 (ablation) | PaDIS · CT, 20 Views | PSNR: 33.36 -> 33.57 | Helped |
| Use patch size 56 rather than patch size 256 (whole-image training) (ablation) | PaDIS · CT, 20 Views | PSNR: 32.84 -> 33.57 | Helped |
| Add positional encoding to patches (ablation) | PaDIS · CT, 20 Views | PSNR: 23.25 -> 33.57 | Helped |
| Add positional encoding to patches (ablation) | PaDIS · CT, 20 Views | SSIM: 0.459 -> 0.854 | Helped |
| Initialize without positional encoding using a baseline image instead of pure noise (ablation) | PaDIS without positional encoding · CT, 20 Views | PSNR: 23.25 -> 24.51 | Helped |
| Use a patch-based rather than whole-image prior (ablation) | CT reconstruction, limited training data · CT, 20 Views | PSNR: 29.12 -> 32.28 | Helped |
Surprising findings
- Positional encoding is essential: without it the patch-based network fails completely (23.25 dB PSNR on CT), even with baseline initialization.
Limitations
- Whole-image model comparison in the limited-data setting may be unfavourable to the baseline, per the authors' own caveat.
- Large patch sizes become infeasible for high-resolution 2D and 3D images due to memory, and the method relies on the number of sampling iterations being large.
Experimental setup
Datasets: AAPM 2016 CT challenge CelebA-HQ
Benchmarks: AAPM 2016 CT challenge (CT reconstruction) CelebA-HQ (deblurring, superresolution)
Metrics: PSNR SSIM
Baselines: Whole image diffusion Langevin dynamics Predictor-corrector VE-DDNM Patch Averaging Patch Stitching ADMM-TV PnP-ADMM PnP-RED FBP baseline
Builds on: DPS Score-based generative modeling (VE-SDE) DDNM Langevin dynamics Predictor-corrector sampling
Closest papers by meaning
- DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction (NeurIPS 2024 · similarity 0.91)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.89)
- Inference-Time Search Using Side Information for Diffusion-Based Image Reconstruction (NeurIPS 2026 · similarity 0.89)
- Prompt-tuning Latent Diffusion Models for Inverse Problems (ICML 2024 · similarity 0.89)
- Diffusion State-Guided Projected Gradient for Inverse Problems (ICLR 2025 · similarity 0.88)
- Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.