Research map / Video / Video segmentation and tracking
Spatiotemporal Blind-Spot Network with Calibrated Flow Alignment for Self-Supervised Video Denoising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STBN, a self-supervised blind-spot video denoiser with bidirectional optical-flow-aligned propagation, spatial receptive field expansion, and unsupervised flow distillation from pseudo-clean frames.
Problem
Self-supervised video denoising struggles with inter-frame redundancy use, limited temporal windows, and biased or noise-sensitive optical flow alignment that conflicts with blind-spot independence assumptions.
Approach
Bidirectional blind-spot feature propagation with flow warping (using nearest-neighbor rather than bilinear interpolation to preserve noise independence), a patch-shuffle-based spatial receptive field expansion module, and distilling a noise-robust optical flow estimator from the model's own initially restored frames.
Key results
- On Set8 with Gaussian noise (avg over sigma 10-50), STBN achieves 33.32 dB PSNR vs RDRF 32.92 dB and ER2R 33.10 dB.
- On DAVIS, STBN averages 36.49 dB PSNR, exceeding RDRF by 1.31 dB.
- On CRVD real raw video, STBN averages 45.77 dB PSNR, surpassing RDRF by 0.68 dB and ER2R by 0.13 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BSA block to temporal propagation (ablation) | STBN · Set8, sigma=30 | PSNR: 32.14 -> 32.49 | Helped |
| Add SRFE module to propagation with BSA (ablation) | STBN · Set8, sigma=30 | PSNR: 32.49 -> 32.68 | Helped |
| Add optical flow refinement to propagation with BSA and SRFE (ablation) | STBN · Set8, sigma=30 | PSNR: 32.68 -> 32.76 | Helped |
| Use bilinear instead of nearest-neighbor interpolation for alignment (ablation) | STBN · Set8 | PSNR | Hurt |
| Tune optical flow refinement coefficient to 5 × 10^{-4} (ablation) | STBN · Set8, hypersmooth video | PSNR | Helped |
| Use PWC-Net rather than DIS for optical flow (ablation) | STBN · Set8, optical flow model ablation | PSNR: 32.28 -> 32.68 | Helped |
| Use PWC-Net rather than SPyNet for optical flow (ablation) | STBN · Set8, optical flow model ablation | PSNR: 32.41 -> 32.68 | Helped |
| Use STBN instead of RDRF | Self-supervised video denoising · Set8, avg | PSNR: 32.92 -> 33.32 | Helped |
| Use STBN instead of RDRF | Self-supervised video denoising · DAVIS, avg | PSNR: 35.18 -> 36.49 | Helped |
| Use STBN instead of RDRF | Self-supervised video denoising · CRVD, avg | PSNR: 45.09 -> 45.77 | Helped |
| Use STBN instead of ER2R_p | Self-supervised video denoising · CRVD, avg | PSNR: 45.64 -> 45.77 | Helped |
| Use STBN_s instead of VRT | Fully self-supervised video denoising · Set8, avg | PSNR: 33.92 -> 34.35 | Helped |
Surprising findings
- Bilinear interpolation in flow warping breaks noise independence and degrades training, while nearest-neighbor warping preserves it.
- The self-supervised model trained on test sequences (Ours_s) surpasses supervised VRT by 0.43 dB on Set8 in the appendix.
Limitations
- Ablation and comparisons rely on Set8/DAVIS/CRVD; generalization to other real-world video noise is not shown.
- Flow refinement adds a distillation hyperparameter (alpha) that must be tuned; the paper reports peak PSNR at 5e-4 only on one video.
Experimental setup
Datasets: DAVIS Set8 CRVD
Benchmarks: Set8 DAVIS CRVD
Metrics: PSNR SSIM
Baselines: VBM4D DVDnet FastDVDnet FloRNN UDVD RDRF ER2R RViDeNet MaskDnGAN VRT RVRT
Builds on: blind-spot network FloRNN RDRF UDVD PWC-Net
Closest papers by meaning
- Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios (CVPR 2024 · similarity 0.85)
- Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising (AAAI 2025 · similarity 0.85)
- Pseudo-Siamese Blind-spot Transformers for Self-Supervised Real-World Denoising (NeurIPS 2024 · similarity 0.85)
- APR-RD: Complemental Two Steps for Self-Supervised Real Image Denoising (AAAI 2025 · similarity 0.83)
- Masked and Shuffled Blind Spot Denoising for Real-World Images (CVPR 2024 · similarity 0.82)
- RivuletMLP: An MLP-based Architecture for Efficient Compressed Video Quality Enhancement (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.