Research map / Image generation / Image restoration
A Principled Self-Referenced Early Stopping Approach for Deep Image Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes self-referenced early stopping for Deep Image Prior using pseudo-reference images (channel similarity, held-out masks, augmented auxiliary channels), with theory on single-reference validation and shared-noise effects, reducing PSNR gap to oracle on denoising, SR and inpainting.
Problem
Early stopping for Deep Image Prior in inverse imaging problems, where overfitting to noise must be detected from a single noisy observation without clean data or accurate noise-level knowledge; variance-based criteria (WMV-ES) stop prematurely on textured or high-frequency images.
Approach
Overfitting detection is cast as validation against a pseudo-reference built from the noisy observation itself: cross-channel similarity for RGB (CSR-ES), held-out pixel masks for grayscale (MR-ES), and auxiliary corrupted copies fitted as extra output channels with a frequency-bias score (ACR-ES).
Key results
- On natural images under Gaussian noise, CSR-ES reduces the average PSNR gap to the oracle to 0.17 dB on Set3/Set14, versus 1.16 dB for SURE-ES and 1.07 dB for WMV-ES.
- On CT10 and T2-20 under Poisson noise, ACR-ES reduces the average gap to the oracle from 4.46 dB to 0.80 dB on CT10 and from 6.41 dB to 1.42 dB on T2-20 relative to WMV-ES.
- On Urban10 inpainting with 50% missing pixels under Gaussian noise, CSR-ES reduces the gap from 3.63 dB (WMV-ES) to 1.16 dB.
Surprising findings
- The variance-minimum heuristic of WMV-ES can fail: for super-resolution, the PSNR peak occurred near a local maximum of the variance curve rather than a minimum.
- ACR-ES uses an argmax of a frequency-bias score instead of an argmin, because the score follows an increase-then-decrease pattern during DIP training.
Limitations
- Theoretical results are for a stylized noise-fitting model and single-reference validation; the pseudo-reference analysis is stated as a design principle rather than a full dynamical theory.
- MR-ES is less stable under Poisson noise because signal-dependent noise makes a held-out mask reference less reliable.
- ACR-ES introduces an auxiliary noise scale hyperparameter tau, and the MR-ES mask requires the held-out region to be representative.
Experimental setup
Datasets: Set3 Set14 Set18 CBSD68 Urban100 Urban10 MRI15 CT10 PD20 T2-20
Benchmarks: Set14 Set18 CBSD68 Urban100 MRI15 CT10 PD20 T2-20 Urban10 (SR x4, x8, inpainting)
Metrics: PSNR PSNR gap to oracle
Baselines: WMV-ES SURE-ES
Builds on: Deep Image Prior WMV-ES SURE
Closest papers by meaning
- Image Reconstruction Via Autoencoding Sequential Deep Image Prior (NeurIPS 2024 · similarity 0.90)
- UGoDIT: Unsupervised Group Deep Image Prior Via Transferable Weights (NeurIPS 2025 · similarity 0.87)
- A Restoration Network as an Implicit Prior (ICLR 2024 · similarity 0.86)
- Dual Prior Unfolding for Snapshot Compressive Imaging (CVPR 2024 · similarity 0.86)
- Stochastic Deep Restoration Priors for Imaging Inverse Problems (ICML 2025 · similarity 0.86)
- Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.