Research map / Video / Action recognition
TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devices
Ji Lin, Chuang Gan, Kuan Wang, Song Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add temporal shift (TSM) to the 2D TSN baseline | TSN ResNet-50 · Something-Something V1 | top-1 accuracy: 20.5 -> 47.3 | Improved |
| add temporal shift (TSM) to the 2D TSN baseline | TSN ResNet-50 · Kinetics | top-1 accuracy: 70.6 -> 74.1 | Improved |
| add TSM to different backbones | MobileNet-V2, ResNet-50, ResNeXt-101, NL ResNet-50 · Kinetics | top-1 accuracy: +3.0, +3.4, +3.9, +1.1 | Improved |
| shift all channels naively (full in-place shift) instead of the 2D baseline (ablation) | naive shift · Kinetics | top-1 accuracy: 2.6% accuracy drop | Hurt |
| shift only a partial proportion of channels (1/8) rather than all (ablation) | TSM · latency overhead | latency overhead: 13.7% -> 3% | Improved |
| insert shift inside residual branch instead of in-place (ablation) | TSM · Kinetics | top-1 accuracy | Improved |
| shift 1/4 of channels (peak) instead of too few or too many (ablation) | residual TSM · Kinetics | top-1 accuracy | Improved |
| use uni-directional (online) instead of bi-directional shift (ablation) | TSM · Something-Something V1 | top-1 accuracy: 47.3 -> 46.3 | Hurt |
| use uni-directional (online) instead of bi-directional shift (ablation) | TSM · Kinetics | top-1 accuracy: 74.1 -> 74.3 | No clear effect |
| replace TSM with 3x1x1 temporal convolution (ablation) | TSM 8f · Something-Something V1 | top-1 accuracy: 45.6 -> 44.9 | Hurt |
| insert online TSM into R-FCN backbone | R-FCN ResNet-101 · ImageNet-VID | mAP: 74.7 -> 76.3 | Improved |
| add optical flow branch to RGB TSM | TSM · Something-Something V1 | top-1 accuracy: 5.4% top-1 improvement | Improved |
Closest papers by meaning
- Gate-Shift Networks for Video Action Recognition (CVPR 2020 · similarity 0.87)
- End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames (CVPR 2024 · similarity 0.86)
- TDN: Temporal Difference Networks for Efficient Action Recognition (CVPR 2021 · similarity 0.86)
- TEINet: Towards an Efficient Architecture for Video Recognition (AAAI 2020 · similarity 0.86)
- Temporal Segment Networks for Action Recognition in Videos (TPAMI 2018 · similarity 0.85)
- Temporal Segment Networks: Towards Good Practices for Deep Action Recognition (ECCV 2016 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
