Research map / Image generation / Image restoration
Using Powerful Prior Knowledge of Diffusion Model in Deep Unfolding Networks for Image Compressive Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion Message Passing (DMP), an AMP-inspired iterative algorithm embedding a pre-trained diffusion model, and unfolds it into DMP-DUN (and DMP-DUN+ with fine-tuned diffusion weights) for image compressive sensing with only 2-4 steps.
Problem
Image compressive sensing reconstruction: DUNs depend on learned priors, while pre-trained diffusion models have strong priors but need many iterative steps and perform poorly at low CS ratios.
Approach
Embed one reverse-diffusion step of a pre-trained diffusion model into each AMP-style message-passing iteration (DMP), then deeply unfold it so lightweight ResBlocks replace the Onsager divergence (Monte-Carlo SURE) and Gaussian-filter components, with learnable time steps and scaling parameters trained end-to-end.
Key results
- DMP-DUN+ (4-step) averages 33.26 dB on Set11 and 31.42 dB on Urban100 across CS ratios, exceeding PRL-RND+ by 1.39 dB and 2.15 dB while using 258.8G fewer FLOPs.
- DMP-DUN+ (2-step) achieves 32.74 dB on Set11 average with 157.0G FLOPs, comparable to CPP-Net's FLOPs but +1.05 dB PSNR.
- On 256x256 crops, DMP-DUN+ (4-step) beats IDM by 1.23 dB average PSNR (37.41 vs 36.28).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DMP-DUN+ (4-step) instead of PRL-RND+ | Image CS reconstruction · Set11 | average PSNR: 1.39dB | Helped |
| use DMP-DUN+ (4-step) instead of PRL-RND+ | Image CS reconstruction · Urban100 | average PSNR: 2.15dB | Helped |
| use DMP-DUN+ instead of IDM | Diffusion-based image CS reconstruction · Urban100 and DIV2K, 256 × 256 center-cropped | average PSNR: 1.23dB | Helped |
| remove Gradient Descent from the DMP step (ablation) | DMP-DUN+ · Set11, Set14, and Urban100 at CS ratio = 10% | PSNR: 33.22/0.9277; 31.71/0.8664; 31.39/0.9053 -> 29.89/0.8713; 28.77/0.8015; 27.27/0.8225 | Hurt |
| remove ResBlocks from the DMP step (ablation) | DMP-DUN+ · Set11, Set14, and Urban100 at CS ratio = 10% | PSNR: 33.22/0.9277; 31.71/0.8664; 31.39/0.9053 -> 32.14/0.9097; 31.04/0.8476; 29.73/0.8702 | Hurt |
| remove the pre-trained diffusion model from the DMP step (ablation) | DMP-DUN+ · Set11, Set14, and Urban100 at CS ratio = 10% | PSNR: 33.22/0.9277; 31.71/0.8664; 31.39/0.9053 -> 31.89/0.9009; 30.82/0.8423; 29.14/0.8627 | Hurt |
| use pre-trained weights at the beginning of training (ablation) | DMP-DUN+ (4-step) · Set11 and Urban100 at CS ratios 4% and 25% | average PSNR: 32.52; 32.10 | Helped |
| increase DMP-DUN iterative steps from 2 to 10 (ablation) | DMP-DUN · Set11, CS ratio = 10% | PSNR: 31.39/0.9030 -> 32.51/0.9161 | Helped |
| increase DMP-DUN iterative steps from 2 to 10 (ablation) | DMP-DUN · Urban100, CS ratio = 10% | PSNR: 28.47/0.8600 -> 30.04/0.8857 | Helped |
| replace Monte-Carlo SURE with ResBlock to approximate the Onsager term (ablation) | DMP-DUN · DMP computation | FLOPs (G), 4 steps: 536.4 -> 24.4 | Helped |
Surprising findings
- Without loading pre-trained diffusion weights, DMP-DUN+ (4-step) drops only 0.42 dB yet still outperforms other SOTA DUNs.
- DDNM (10-step) performs very poorly at low CS ratios (e.g., about 12-13 dB at 1-4%).
Limitations
- Theoretical state-evolution derivation relies on assumptions (perfect denoiser, perfect Gaussian filter) that are hard to meet in practice.
- Divergence/Onsager term and Gaussian filter are approximated by lightweight learned ResBlocks rather than exact computation.
- DMP-DUN+ fine-tuning of the diffusion model significantly increases training cost.
Experimental setup
Datasets: COCO2017 BSDS500 ImageNet Set11 Urban100 DIV2K
Benchmarks: Set11 Urban100 DIV2K Set14
Metrics: PSNR SSIM FLOPs Params
Baselines: ISTA-Net+ AMP-Net TransCS CSformer DPC-DUN OCTUF PRL-RND+ CAT-Net UFC-Net CPP-Net DDNM DDNM+
Builds on: Approximate Message Passing DDIM Deep unfolding networks DDNM Monte-Carlo SURE
Closest papers by meaning
- CPP-Net: Embracing Multi-Scale Feature Fusion into Deep Unfolding CP-PPA Network for Compressive Sensing (CVPR 2024 · similarity 0.89)
- HUNet: Homotopy Unfolding Network for Image Compressive Sensing (CVPR 2025 · similarity 0.88)
- UFC-Net: Unrolling Fixed-point Continuous Network for Deep Compressive Sensing (CVPR 2024 · similarity 0.87)
- Dual Prior Unfolding for Snapshot Compressive Imaging (CVPR 2024 · similarity 0.87)
- Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms (NeurIPS 2024 · similarity 0.87)
- Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.