Research map / Video / Video segmentation and tracking
Spatio-temporal Transformer Network for Video Restoration
Tae Hyun Kim, Mehdi S. M. Sajjadi, Michael Hirsch, Bernhard Scholkopf
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the reference frame as an extra sampling target in the spatio-temporal sampler (ablation) | STTN · Video SR reconstruction (2 input frames) | MSE: 17.34 -> 13.01 | Improved |
| add the reference frame as an extra sampling target (ablation) | STTN · Video deblurring reconstruction (2 input frames) | MSE: 8.7 -> 5.51 | Improved |
| add the reference frame as an extra sampling target (ablation) | STTN · Video SR reconstruction (5 input frames) | MSE: 14.39 -> 12.38 | Improved |
| increase the number of input frames from 2 to 5 (with reference frame) (ablation) | STTN · Video SR reconstruction | MSE: 13.01 -> 12.38 | Improved |
| increase the number of input frames from 2 to 5 (with reference frame) (ablation) | STTN · Video deblurring reconstruction | MSE: 5.51 -> 5.42 | Mixed |
| add STTN with one spatio-temporal transformer to VDSR | VDSR · Video SR (own YouTube dataset, x4) | PSNR (dB): 28.63 -> 28.94 | Improved |
| use two spatio-temporal transformers instead of one | STTN+VDSR · Video SR (own YouTube dataset, x4) | PSNR (dB): 28.94 -> 29.10 | Improved |
| replace two spatial transformers with one spatio-temporal transformer | VDSR · Video SR (own YouTube dataset, x4) | PSNR (dB): 28.85 -> 28.94 | Improved |
| integrate STTN into FRVSR | FRVSR · Video SR (own YouTube dataset, x4) | PSNR (dB): 29.78 -> 30.09 | Improved |
| integrate STTN into FRVSR | FRVSR · Video SR (own YouTube dataset, x4) | parameters: ~2.6M -> ~2.4M | Improved |
| add STTN in front of DVD, replacing four optical flow computations | DVD · Video deblurring (DVD dataset) | PSNR (dB): 31.02 -> 31.45 | Improved |
| add STTN in front of DVD | DVD · Video deblurring (DVD dataset) | speed (FPS): 0.04 -> 1.99 | Improved |
Closest papers by meaning
- Spatio-Temporal Filter Adaptive Network for Video Deblurring (ICCV 2019 · similarity 0.90)
- Real-Time Video Super-Resolution With Spatio-Temporal Networks and Motion Compensation (CVPR 2017 · similarity 0.89)
- Blur-aware Spatio-temporal Sparse Transformer for Video Deblurring (CVPR 2024 · similarity 0.88)
- FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring (CVPR 2024 · similarity 0.87)
- Spatial-Temporal Transformer for Video Snapshot Compressive Imaging (TPAMI 2022 · similarity 0.87)
- StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
