Research map / Video / Video segmentation and tracking
STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond
Zheng Chang, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Wen Gao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add temporal attention module (T_AMI) (ablation) | STAU · Moving MNIST | MSE: 29.9 -> 27.1 | Improved |
| add temporal attention module (T_AMI) (ablation) | STAU · Moving MNIST | SSIM: 0.933 -> 0.939 | Improved |
| add temporal-from-spatial attention (S->T) (ablation) | STAU · Moving MNIST | MSE: 31.2 -> 29.6 | Improved |
| add spatial-from-temporal attention (T->S) (ablation) | STAU · Moving MNIST | MSE: 31.2 -> 29.5 | Improved |
| use both S->T and T->S correlations (ablation) | STAU · Moving MNIST | MSE: 29.5 -> 27.1 | Improved |
| increase temporal and spatial receptive field to tau=5, theta=5 (ablation) | STAU · Moving MNIST | MSE: 33.4 -> 27.1 | Improved |
| increase temporal receptive field tau from 1 to 5 at theta=1 (ablation) | STAU · Moving MNIST | MSE: 33.4 -> 29.5 | Improved |
| increase spatial receptive field theta from 1 to 5 at tau=1 (ablation) | STAU · Moving MNIST | MSE: 33.4 -> 29.9 | Improved |
| replace MAU unit with STAU unit (ablation) | video prediction model · Moving MNIST | MSE: 29.5 -> 27.1 | Improved |
| add STAU predictive unit as video predictor before classification (ablation) | classifier · Something-SomethingV2 front 25% | top-1: 2.10 -> 7.50 | Improved |
| use STAU instead of MAU (ablation) | early action recognition · Something-SomethingV2 front 50% | top-1: 10.60 -> 12.90 | Improved |
| use STAU instead of MAU (ablation) | parameters and inference time · Moving MNIST | inference time (s/800 samples): 17.34 -> 19.58 | Hurt |
Closest papers by meaning
- Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning (CVPR 2023 · similarity 0.89)
- MAU: A Motion-Aware Unit for Video Prediction and Beyond (NeurIPS 2021 · similarity 0.88)
- Collaborative Spatiotemporal Feature Learning for Video Action Recognition (CVPR 2019 · similarity 0.88)
- STM: SpatioTemporal and Motion Encoding for Action Recognition (ICCV 2019 · similarity 0.88)
- A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition (ICCV 2023 · similarity 0.87)
- Wavelet-Driven Spatiotemporal Predictive Learning: Bridging Frequency and Time Variations (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
