Research map / Image generation / Image restoration
Image Reconstruction Via Autoencoding Sequential Deep Image Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes aSeqDIP, a data-free deep image prior that sequentially updates U-Net weights with an input-adaptive data-consistency loss plus an autoencoding regularizer, reducing noise overfitting for MRI, CT, denoising, inpainting and deblurring.
Problem
Deep Image Prior (DIP) overfits noise because of network overparameterization, so the number of optimization iterations must be carefully chosen; diffusion-based alternatives need large clean training data or pre-trained models.
Approach
Replace the fixed random DIP input with a sequential scheme: each U-Net stage is initialized from the previous one, optimized with data consistency plus an autoencoding term that ties its output to its input, and the input is then updated by a single forward pass, with no input gradient updates.
Key results
- On 4x MRI (setting 1), aSeqDIP reaches 34.08 dB PSNR vs 33.6 dB for Self-Guided DIP and 31.51 dB for Score-MRI, at about 2.2 minutes.
- On 30-view CT, aSeqDIP achieves 33.09 dB and SSIM 0.92 at 2.2 minutes, versus 31.35 dB for MCG (6.4 minutes).
- Denoising and in-painting PSNR gains of about 0.6 dB over DPS, with aSeqDIP run-time slightly lower on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use aSeqDIP instead of Self-Guided DIP | MRI reconstruction · fastMRI 4x and 8x | PSNR (dB): 33.6±0.23 -> 34.08±0.41; 30.75±0.25 -> 31.34±0.47 | Helped |
| Use aSeqDIP instead of Self-Guided DIP | CT reconstruction · AAPM CT 18 and 30 views | PSNR (dB): 33.95±0.32 -> 34.88±0.36; 31.95±0.32 -> 33.09±0.39 | Helped |
| Use aSeqDIP instead of the strongest listed baseline | CBSD68 denoising · σ=15 and σ=30 | PSNR (dB): 31,21±0.26 -> 31.51±0.34; 28.67±0.25 -> 28.97±0.44 | Helped |
| Use aSeqDIP instead of DPS | CBSD68 in-painting · HIAR=0.1 and 0.25 | PSNR (dB): 23.9±0.45 -> 24.56±0.45; 22.03±0.36 -> 22.57±0.47 | Helped |
| Use aSeqDIP instead of DPS | CBSD68 non-linear deblurring · BKSE | PSNR (dB): 23.40±0.56 -> 23.89±0.40 | Helped |
| Add the autoencoding regularization term (λ=1) rather than disable it (λ=0) (ablation) | aSeqDIP · MRI 4x and CT 18 views | PSNR (dB) | Helped |
| Set λ=1 rather than λ=0.5 or λ=2 (ablation) | aSeqDIP · MRI 4x | PSNR (dB) | Mixed |
| Use N=2, K=2000 rather than N=1, K=4000 or N=4, K=1000 (NK=4000) (ablation) | aSeqDIP · MRI, CT, denoising, and in-painting | PSNR (dB) | Helped |
| Use an autoencoding term rather than omit it (ablation) | aSeqDIP · MRI 4x and CT 18 views | PSNR (dB) | Helped |
| Train a clean-image autoencoder and optimize its input at test time (ablation) | MRI reconstruction · fastMRI 4x | PSNR (dB) | Helped |
Surprising findings
- Using a noisy copy of the ground truth as DIP input improves reconstruction, motivating input-adaptive updates.
- Without the autoencoding term, DIP-style sequential updates still overfit noise, though the term delays the onset of overfitting.
Limitations
- Noise overfitting still eventually occurs even with the autoencoding regularizer, as the authors acknowledge.
- Per-instance optimization requires re-running for each new measurement set, unlike end-to-end supervised models such as VarNet, which score higher on MRI (34.89 dB with 8000 training scans).
- aSeqDIP underperforms DDNM on box in-painting on FFHQ by under 1 dB.
Experimental setup
Datasets: fastMRI AAPM CBSD68 FFHQ
Benchmarks: fastMRI AAPM CBSD68 FFHQ
Metrics: PSNR SSIM run-time
Baselines: Vanilla DIP Self-Guided DIP ES-DIP Ref-Guided DIP TV-DIP SGLD-DIP Score-MRI MCG DPS Rethinking-DIP
Builds on: Deep Image Prior Self-Guided DIP Score-MRI Manifold Constrained Gradient Diffusion Posterior Sampling
Closest papers by meaning
- A Principled Self-Referenced Early Stopping Approach for Deep Image Prior (NeurIPS 2026 · similarity 0.90)
- UGoDIT: Unsupervised Group Deep Image Prior Via Transferable Weights (NeurIPS 2025 · similarity 0.89)
- A Restoration Network as an Implicit Prior (ICLR 2024 · similarity 0.87)
- LMD: Faster Image Reconstruction with Latent Masking Diffusion (AAAI 2024 · similarity 0.87)
- Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise (NeurIPS 2024 · similarity 0.86)
- Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.