Research map / Video / Video segmentation and tracking
Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Develops a theoretical framework for untrained-network (DIP) snapshot compressive imaging recovery, optimizes Bernoulli mask probability, and proposes SCI-BDVP, a bagged deep video prior PGD algorithm that is state-of-the-art among UNN video SCI methods.
Problem
Recovering 3D video/hyperspectral data cubes from a single 2D snapshot compressive imaging measurement without training data, with theoretical understanding of how mask design and untrained network size affect recovery under noise-free and noisy measurements.
Approach
Provides error bounds for DIP-based SCI recovery linking the number of recoverable frames and DIP parameter count to the mask non-zero probability p, and extends bagged-DIP to video via a PGD scheme with multiple patch-size deep video priors averaged and a skip connection.
Key results
- Noise-free video SCI: SCI-BDVP (GAP) averages 29.41 dB PSNR / 0.882 SSIM, about 1 dB above other untrained methods (Factorized-DVP 28.21 dB), but below PnP-FastDVDnet's 32.54 dB.
- Noisy measurements (sigma=10/25/50): SCI-BDVP (GD) averages 27.73/27.23/24.21 dB, outperforming the supervised FastDVD (GAP) at sigma=25 and 50 averages only in some settings; PnP-DIP averages 24.49/24.19/21.17 dB.
- Theory suggests optimal mask probability p* < 0.5 for noise-free measurements, while p* increases with noise level and can exceed 0.5 under noise.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add skip connection (weighted average of GD and projection outputs) (ablation) | SCI-BDVP · noise-free video SCI | PSNR (dB): about 4 dB drop without skip connection | Helped |
| remove bagging (single DVP projection instead of bagged DVP) (ablation) | SCI-BDVP · noise-free and noisy video SCI | PSNR (dB): bagging improves reconstruction quality in both noise-free and noisy cases (qualitative/figure) | Helped |
| use bagged-DVP projection with GAP descent instead of prior UNN methods | noise-free video SCI · 6 benchmark videos, B=8 | PSNR (dB): about 1 dB average PSNR gain over other untrained methods (29.41 vs 28.21 for Factorized-DVP) | Helped |
| replace GAP descent with GD descent for noisy measurements | SCI-BDVP · noisy video SCI, sigma=10/25/50 | PSNR (dB): SCI-BDVP (GD) averages 27.73/27.23/24.21 vs 25.25/21.15/16.27 for E2E | Helped |
| optimized mask probability (p in 0.5-0.7) instead of Bern(0.5) mask (ablation) | SCI-BDVP (PGD) · noisy video SCI, sigma=50 | PSNR (dB): 24.21 -> 25.24 average PSNR (about 1 dB) | Helped |
| optimized mask probability instead of Bern(0.5) mask (ablation) | SCI-BDVP (GAP) · noise-free video SCI | PSNR (dB): 29.41 -> 29.95 average PSNR | Helped |
| mask non-zero probability p (noise-free) | SCI-BDVP (GAP) and PnP-FastDVDnet (GAP) · noise-free video SCI | PSNR (dB) | No clear effect |
| GAP descent instead of GD for noise-free measurements | SCI-BDVP · noise-free video SCI | convergence | Helped |
| add measurement-loss regularizer (omega>0) in DVP training (ablation) | SCI-BDVP · noisy video SCI (sigma=25) | PSNR (dB) | Helped |
| increase measurement-loss coefficient omega (ablation) | SCI-BDVP · noise-free video SCI | PSNR (dB) | No clear effect |
| increase measurement-loss coefficient omega beyond the chosen value (ablation) | SCI-BDVP · noisy video SCI (sigma=25) | PSNR (dB) | Hurt |
| mask optimization with SCI-BDVP (noisy case): optimal p increases with noise level | mask probability p · noisy video SCI | PSNR (dB) | Mixed |
Surprising findings
- For noisy measurements the optimal mask non-zero probability can exceed 0.5, contrary to noise-free theory.
- PnP-FastDVDnet (GAP) reaches higher noise-free PSNR than all untrained methods, beating SCI-BDVP by about 3 dB on average.
Limitations
- Theory only covers i.i.d. Bernoulli masks, not practical constrained hardware masks.
- Experiments limited to a small set of classic grayscale benchmark videos and additive Gaussian noise.
- Runtime per video is tens of minutes per reconstruction.
Experimental setup
Datasets: Kobe Runner Drop Traffic Aerial Vehicle Crash
Benchmarks: Kobe Traffic Runner Drop Crash Aerial
Metrics: PSNR SSIM
Baselines: GAP-TV PnP-FFDNet PnP-FastDVDnet PnP-DIP Factorized-DVP
Builds on: deep image prior bagged-DIP GAP-TV Factorized DVP deep decoder
Closest papers by meaning
- Memory-Efficient Network for Large-Scale Video Compressive Sensing (CVPR 2021 · similarity 0.91)
- SCINeRF: Neural Radiance Fields from a Snapshot Compressive Image (CVPR 2024 · similarity 0.88)
- UGoDIT: Unsupervised Group Deep Image Prior Via Transferable Weights (NeurIPS 2025 · similarity 0.88)
- Dual Prior Unfolding for Snapshot Compressive Imaging (CVPR 2024 · similarity 0.87)
- NPN: Non-Linear Projections of the Null-Space for Imaging Inverse Problems (NeurIPS 2025 · similarity 0.87)
- CPP-Net: Embracing Multi-Scale Feature Fusion into Deep Unfolding CP-PPA Network for Compressive Sensing (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.