Research map / Image generation / Image restoration
Masked Pre-training Enables Universal Zero-shot Denoiser
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MPI, which masked-pre-trains a DIP-style hourglass network on ImageNet and then iteratively fills masked pixels of a single noisy image with EMA ensembling for fast zero-shot denoising.
Problem
Zero-shot image denoising methods train from scratch on each noisy image, which is slow and prone to overfitting or blur; the paper asks whether natural-image knowledge from pre-training can improve zero-shot denoising quality and speed.
Approach
Pre-train a U-shaped hourglass network (DIP architecture) on ImageNet with pixel-wise random masking to reconstruct masked pixels, then at test time fine-tune it on a single noisy image with the same masked-reconstruction objective, keeping only masked-area predictions and aggregating them via EMA across iterations.
Key results
- On CSet Gaussian sigma=25, MPI reaches 31.61/0.841 PSNR/SSIM versus 30.42/0.795 for DIP, with about 51.6 s average inference versus 451.9 s for DIP (the faster variant takes 10.1 s at 30.83/0.824).
- On Kodak generalization across six unseen noise settings, MPI averages 33.30/0.890, the best among the compared supervised, unsupervised and zero-shot methods.
- On SIDD validation, MPI achieves 34.43/0.844 PSNR/SSIM, outperforming FasterDIP (33.55/0.795) and DIP (33.68/0.802) among zero-shot methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MPI instead of DIP | Zero-shot denoising · CSet, Gaussian σ=25 | PSNR: 30.42/0.795 -> 31.61/0.841 | Helped |
| use MPI instead of ZS-N2N | Zero-shot denoising · CSet, Poisson λ=25 | PSNR: 28.06/0.711 -> 30.02/0.785 | Helped |
| use MPI instead of ZS-N2N | Zero-shot denoising · Kodak, average across 6 noise settings | PSNR: 31.17/0.823 -> 33.30/0.890 | Helped |
| use pre-trained weights rather than training from scratch (ablation) | MPI · CSet, Gaussian σ=25 | PSNR: 30.90/0.811 -> 31.61/0.841 | Helped |
| use pre-trained weights rather than training from scratch (ablation) | MPI · SIDD validation | PSNR: 32.31/0.746 -> 34.43/0.844 | Helped |
| use pre-trained weights rather than training from scratch (ablation) | MPI · FMD | PSNR: 31.44/0.786 -> 32.97/0.847 | Helped |
| use EMA ensemble rather than no ensemble (ablation) | MPI · CSet, Gaussian σ=25 | PSNR: 30.23/0.797 -> 31.61/0.841 | Helped |
| remove masking from the loss and ensemble (ablation) | MPI · CSet, Gaussian σ=25 | PSNR: 31.61/0.841 -> 23.48/0.441 | Hurt |
| use only the final forward prediction instead of EMA ensemble (ablation) | MPI · CSet, Gaussian σ=25 | PSNR: 31.61/0.841 -> 13.73/0.154 | Hurt |
| add early stopping (ablation) | MPI · CSet, Gaussian σ=25 | PSNR: 31.61 -> 31.66 | Helped |
| add Gaussian σ=25 noise during pre-training (ablation) | MPI · CSet, Gaussian σ=25 | PSNR: 31.61 -> 31.24 | Hurt |
| replace pixel-shuffle downsampling with RSG (ablation) | MPI · SIDD validation, 256×256 | PSNR: 34.42/0.843 -> 34.75/0.852 | Helped |
Surprising findings
- A model pre-trained only with masked reconstruction, combined with simple ensembling and no noise assumptions, already denoises unseen noise types before any per-image optimization.
- Adding synthetic Gaussian noise to the pre-training data (noise-to-noise) lowered performance (31.24 vs 31.61 PSNR), suggesting noise assumptions in pre-training hurt.
Limitations
- Overfitting to noise still occurs with more iterations, requiring early stopping.
- Real-world denoising needs high masking ratios (80-95%) and pixel-shuffle downsampling, with per-dataset tuning of hyperparameters.
- Inference still takes tens of seconds to about a minute per image for the default setting.
Experimental setup
Datasets: ImageNet CSet McMaster CBSD Kodak SIDD PolyU FMD
Benchmarks: CSet McMaster CBSD Kodak SIDD PolyU FMD
Metrics: PSNR SSIM average inference time
Baselines: DIP Noise2Void (single-image) Noise2Self (single-image) ZS-N2N FasterDIP DIP-SURE DDNM DDPG AP-BSN MM-BSN PUCA SwinIR
Builds on: Deep Image Prior Masked Autoencoders Noise2Void Noise2Self Zero-Shot Noise2Noise
Closest papers by meaning
- Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework (NeurIPS 2025 · similarity 0.90)
- Zero-Shot Image Denoising via Hybrid Prior-Guided Pseudo Sample Generation (CVPR 2026 · similarity 0.89)
- Universal Denoising Networks : A Novel CNN Architecture for Image Denoising (CVPR 2018 · similarity 0.89)
- Zero-Shot Noise2Mean: Gap Minimization for Efficient Denoising from a Single Noisy Image (AAAI 2025 · similarity 0.88)
- Zero-Shot Blind-spot Image Denoising via Implicit Neural Sampling (CVPR 2025 · similarity 0.87)
- ZERO-IG: Zero-Shot Illumination-Guided Joint Denoising and Adaptive Enhancement for Low-Light Images (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.