Research map / Video / Video segmentation and tracking
Efficient Online Training for Zero-Shot Time-Lapse Microscopy Denoising and Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MDSR-Zero, a zero-shot online training method for joint denoising and 2x super-resolution of time-lapse microscopy videos that reuses and EMA-averages weights across frames plus a new SR-aware self-supervised loss.
Problem
Zero-shot microscopy denoising processes video frames independently, causing long training times, temporal noise and over-smoothing.
Approach
Efficient Online Training treats the video as a stream: each frame's network is initialized from the previous frame's weights and inference uses an exponential moving average of weights across frames; a denoising-on-super-resolution loss adds a higher-resolution neighbor-pixel self-supervision term to recover detail.
Key results
- Averaged over noise types and levels on CTC, the method reaches 35.42 dB PSNR / 0.9348 SSIM, the best among zero-shot methods and comparable to self-supervised UDVD (35.30) and DeepCAD-RT (35.30).
- EOT adds about 2 dB PSNR over frame-by-frame training in the ablation (e.g., Gaussian 33.27 to 35.33 with LSR).
- Per-frame time is about 2.1 s for 100 iterations versus 6.2 s for N2F with the same network, and the method reports up to 10x training time reduction over N2F.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MDSR-Zero instead of the strongest listed baseline, DeepCAD-RT | CTC PhC-C2DH-U373 mixed Poisson-Gaussian noise · CTC PhC-C2DH-U373 mixed Poisson-Gaussian avg. | PSNR: 35.73 -> 36.77 | Helped |
| Use MDSR-Zero instead of DeepCAD-RT | DeepSeMi EGFP · DeepSeMi EGFP | PSNR: 26.95 -> 27.29 | Helped |
| Add SR self-supervision to LR self-supervision (ablation) | MDSR-Zero · CTC PhC-C2DH-U373 Gaussian noise | PSNR: 31.97 -> 35.20 | Helped |
| Add SR self-supervision to LR self-supervision (ablation) | MDSR-Zero · CTC PhC-C2DH-U373 Poisson noise | PSNR: 32.58 -> 35.34 | Helped |
| Add EOT to LR self-supervision (ablation) | MDSR-Zero · CTC PhC-C2DH-U373 Gaussian noise | PSNR: 31.97 -> 33.27 | Helped |
| Add EOT to LR self-supervision (ablation) | MDSR-Zero · CTC PhC-C2DH-U373 Poisson noise | PSNR: 32.58 -> 33.37 | Helped |
| Increase training iterations from 10 to 200 (ablation) | MDSR-Zero · CTC PhC-C2DH-U373 Poisson noise | PSNR: 33.78 -> 35.42 | Helped |
| Use α = 0.9 for EOT EMA decay (ablation) | MDSR-Zero on the PhC dataset · PhC-C2DH-U373 | PSNR | Helped |
| Use MDSR-Zero rather than frame-by-frame training (ablation) | Zero-shot denoising on video · CTC | PSNR | Helped |
| Use MDSR-Zero rather than frame-by-frame training (ablation) | Zero-shot denoising on video · CTC | training time: 10x | Helped |
| Use SR self-supervision rather than LR self-supervision alone (ablation) | MDSR-Zero · Microscopy images | high-frequency detail recovery | Helped |
Surprising findings
- Naive ZS-N2N trained with the same network and iterations drops to 26.15 dB PSNR, far below its default setting.
Limitations
- EMA decay alpha must be tuned to object motion; performance depends on hyperparameter choice.
- Super-resolution is limited to 2x and assumes bilinear PSF approximation in synthetic experiments.
- Self-supervised baselines are trained on full sequences, so time comparisons are not directly comparable.
Experimental setup
Datasets: Cell Tracking Challenge PhC-C2DH-U373 DeepSeMi dataset (T4, EGFP) ZSDeconv dataset
Benchmarks: CTC PhC-C2DH-U373 synthetic noise DeepSeMi T4/EGFP ZSDeconv Lysosome
Metrics: PSNR SSIM training and inference time per frame parameter count
Baselines: UDVD DeepCAD-RT DeepSeMi Noise2Fast ZS-N2N Neighbor2Neighbor Noise2Self ZSDeconv
Builds on: Noise2Fast Neighbor2Neighbor Noise2Noise Exponential Moving Average PixelShuffle
Closest papers by meaning
- Zero-shot Denoising via Neural Compression: Theoretical and algorithmic framework (NeurIPS 2025 · similarity 0.87)
- Masked Pre-training Enables Universal Zero-shot Denoiser (NeurIPS 2024 · similarity 0.86)
- DRACO: A Denoising-Reconstruction Autoencoder for Cryo-EM (NeurIPS 2024 · similarity 0.86)
- Zero-shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model (AAAI 2025 · similarity 0.85)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.85)
- Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.