Research map / Video / Action recognition
Shrinking Temporal Attention in Transformers for Video Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add short-term attention to current attention (ablation) | STAT · Kinetics400 | Top-1 accuracy: 77.3 -> 77.8 | Helped |
| Add long-term attention to current attention (ablation) | STAT · Kinetics400 | Top-1 accuracy: 77.3 -> 78.3 | Helped |
| Use both short-term and long-term attention with current attention (ablation) | STAT · Kinetics400 | Top-1 accuracy: 77.3 -> 78.6 | Helped |
| Increase short-term attention window from 3 to 5 (ablation) | STAT · Kinetics400 | Top-1 accuracy: 79.9 -> 80.4 | Helped |
| Increase short-term attention window from 5 to 9 (ablation) | STAT · Kinetics400 | Top-1 accuracy: 80.4 -> 80.2 | Hurt |
| Use a spatial class token instead of spatial average pooling (ablation) | STAT · Kinetics400 | Top-1 accuracy: 77.4 -> 78.6 | Helped |
| Add one temporal aggregation layer instead of average pooling (ablation) | STAT · Kinetics400 | Top-1 accuracy: 75.7 -> 78.6 | Helped |
| Increase temporal aggregation layers from 1 to 2 (ablation) | STAT · Kinetics400 | Top-1 accuracy: 78.6 -> 78.6 | No clear effect |
| Increase embedding convolutional layers from 1 to 3 (ablation) | STAT · Kinetics400 | Top-1 accuracy: 78.2 -> 78.6 | Helped |
| Increase embedding convolutional layers from 3 to 5 (ablation) | STAT · Kinetics400 | Top-1 accuracy: 78.6 -> 78.3 | Hurt |
| Use STAT instead of MViT at 64 frames | video action recognition models · Kinetics400 validation | Top-1 accuracy: 81.2 -> 81.3 | Helped |
| Use STAT instead of MViT at 64 frames | video action recognition models · Something-Something V2 validation | Top-1 accuracy: 67.7 -> 67.6 | Hurt |
Closest papers by meaning
- Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers (NeurIPS 2021 · similarity 0.89)
- TEA: Temporal Excitation and Aggregation for Action Recognition (CVPR 2020 · similarity 0.88)
- Time Is MattEr: Temporal Self-supervision for Video Transformers (ICML 2022 · similarity 0.88)
- CAST: Cross-Attention in Space and Time for Video Action Recognition (NeurIPS 2023 · similarity 0.88)
- Adapting Short-Term Transformers for Action Detection in Untrimmed Videos (CVPR 2024 · similarity 0.87)
- Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoders (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.