Research map / Video / Action recognition
Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace joint space-time attention with trajectory attention (ablation) | ViT-B video transformer · SSv2 | top-1 accuracy: 64.0 -> 66.5 | Helped |
| replace joint space-time attention with trajectory attention (ablation) | ViT-B video transformer · K-400 | top-1 accuracy: 79.2 -> 79.7 | Helped |
| normalize softmax per frame instead of over space-time in trajectory attention (ablation) | Motionformer · SSv2 | top-1 accuracy: 60.9 -> 66.5 | Helped |
| use 1D temporal attention over trajectories instead of average pooling (ablation) | Motionformer · SSv2 | top-1 accuracy: 60.0 -> 66.5 | Helped |
| use cubic instead of square tokenization (ablation) | Motionformer · SSv2 | top-1 accuracy: 65.8 -> 66.5 | Helped |
| use separate instead of joint space-time positional encoding (ablation) | joint space-time attention · SSv2 | top-1 accuracy: 60.8 -> 64.0 | Helped |
| use separate instead of joint positional encoding (ablation) | Motionformer · SSv2 | top-1 accuracy: 65.8 -> 66.5 | Helped |
| select prototypes by greedy orthogonality instead of segment-means (ablation) | Orthoformer · SSv2 | top-1 accuracy: 60.3 -> 63.8 | Helped |
| share prototypes across time (ablation) | Orthoformer · SSv2 | top-1 accuracy: 61.5 -> 63.8 | Helped |
| increase prototypes from 16 to 128 (ablation) | Orthoformer · SSv2 | top-1 accuracy: 59.2 -> 63.8 | Helped |
| use Orthoformer instead of Performer for approximate trajectory attention (ablation) | Motionformer · SSv2 | top-1 accuracy: 52.7 -> 63.8 | Helped |
| use Orthoformer instead of Nystromformer (ablation) | Motionformer · SSv2 | top-1 accuracy: 64.0 -> 63.8 | No clear effect |
Closest papers by meaning
- TrAction: Action Recognition with Sparse Trajectories (NeurIPS 2026 · similarity 0.87)
- Understanding Video Transformers via Universal Concept Discovery (CVPR 2024 · similarity 0.87)
- Trajectory attention for fine-grained video motion control (ICLR 2025 · similarity 0.86)
- Video Motion Transfer with Diffusion Transformers (CVPR 2025 · similarity 0.86)
- DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control (NeurIPS 2026 · similarity 0.86)
- Prototypical Transformer As Unified Motion Learners (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.