Research map / Video / Action recognition
VidTr: Video Transformer Without Convolutions
Yanyi Zhang, Xinyu Li, Chunhui Liu, Bing Shuai, Yi Zhu, Biagio Brattoli, Hao Chen, Ivan Marsic, Joseph Tighe
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use square 1x16^2 patches instead of cubic 4x16^2 patches (ablation) | VidTr-S · Kinetics-400 | top-1 accuracy: 73.1 -> 77.7 | Improved |
| use 16x16 patches instead of 32x32 patches (ablation) | VidTr-S · Kinetics-400 | top-1 accuracy: 71.2 -> 77.7 | Improved |
| add temporal attention to spatial-only transformer (WH to WH + T) (ablation) | VidTr-S · Kinetics-400 | top-1 accuracy: 74.7 -> 77.7 | Improved |
| separable attention (WH + T) instead of joint spatio-temporal attention (WHT) (ablation) | VidTr-S · Kinetics-400 | memory / top-1 accuracy: 7.6GB -> 2.3GB memory, 77.5 -> 77.7 | Improved |
| axial attention (W + H + T) instead of WH + T (ablation) | VidTr-S · Kinetics-400 | top-1 accuracy: 77.7 -> 72.3 | Hurt |
| STD topK pooling instead of temporal average pooling (ablation) | C-VidTr-S · Kinetics-400 | top-1 accuracy: 74.9 -> 75.7 | Improved |
| STD topK pooling instead of 1D strided convolution (ablation) | C-VidTr-S · Kinetics-400 | top-1 accuracy: 75.4 -> 75.7 | Improved |
| start temporal down-sampling at layer 1 instead of layer 0 (ablation) | C-VidTr-S · Kinetics-400 | top-1 accuracy: 72.9 -> 74.9 | Improved |
| skip one layer between down-sample layers instead of none (ablation) | C-VidTr-S · Kinetics-400 | top-1 accuracy: 73.9 -> 74.9 | Improved |
| skip more layers between down-sample layers (down-sample later) (ablation) | C-VidTr-S · Kinetics-400 | top-1 accuracy / FLOPs: 74.9 -> 75.2 | No clear effect |
| initialize from ViT-L instead of ViT-B (ablation) | VidTr-S · Kinetics-400 | top-1 accuracy: 77.7 -> 77.5 | No clear effect |
| ensemble VidTr with I3D50 | VidTr-S · Kinetics-400 | top-1 accuracy: 77.7 -> 79.4 | Improved |
Closest papers by meaning
- ViViT: A Video Vision Transformer (ICCV 2021 · similarity 0.90)
- Patch-based Separable Transformer for Visual Recognition (TPAMI 2022 · similarity 0.87)
- Space-time Mixing Attention for Video Transformer (NeurIPS 2021 · similarity 0.87)
- Video Transformers: A Survey (TPAMI 2023 · similarity 0.87)
- TGBFormer: Transformer-GraphFormer Blender Network for Video Object Detection (AAAI 2025 · similarity 0.86)
- Adapting Short-Term Transformers for Action Detection in Untrimmed Videos (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
