Research map / Video / Video segmentation and tracking
Stand-Alone Inter-Frame Attention in Video Models
Fuchen Long, Zhaofan Qiu, Yingwei Pan, Ting Yao, Jiebo Luo, Tao Mei
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SIFA temporal modeling to ResNet-50 | ResNet-50 · Kinetics-400 validation | Top-1 accuracy: 72.0 -> 77.4 | Improved |
| add inter-frame attention over local regions (ablation) | SIFA-C · Kinetics-400 validation | Top-1 accuracy: 73.3 -> 74.6 | Improved |
| add deformable feature resampling (ablation) | SIFA-R · Kinetics-400 validation | Top-1 accuracy: 74.6 -> 75.4 | Improved |
| aggregate deformable neighbors from both previous and next frames (ablation) | SIFA · Kinetics-400 validation | Top-1 accuracy: 0.1% in top-1 accuracy | Improved |
| predict deformable offsets from next-frame features (ablation) | SIFA-R · Kinetics-400 validation | Top-1 accuracy: 74.6 -> 74.7 | Improved |
| predict offsets using 3D convolution over the clip (ablation) | SIFA-R · Kinetics-400 validation | Top-1 accuracy: 74.7 -> 74.8 | Improved |
| predict offsets from temporal differences (ablation) | SIFA-R · Kinetics-400 validation | Top-1 accuracy: 74.8 -> 75.0 | Improved |
| use the motion saliency map to predict offsets (ablation) | SIFA · Kinetics-400 validation | Top-1 accuracy: 75.0 -> 75.4 | Improved |
| expand local attention from 1×1 to 3×3 (ablation) | SIFA · Kinetics-400 validation | Top-1 accuracy: 73.4 -> 75.4 | Improved |
| increase local region size beyond 3×3 (ablation) | SIFA · Kinetics-400 validation | Top-1 accuracy: 75.4; 75.4; 75.5 | No clear effect |
| add SIFA blocks in res3 and res4 (ablation) | SIFA-Net with a block in res5 · Kinetics-400 validation | Top-1 accuracy: 75.4 -> 77.4 | Improved |
| add a SIFA block in res2 (ablation) | SIFA-Net with SIFA in res3–res5 · Kinetics-400 validation | Top-1 accuracy: 77.4 -> 77.4 | No clear effect |
Closest papers by meaning
- Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation (CVPR 2023 · similarity 0.87)
- Video Frame Interpolation Transformer (CVPR 2022 · similarity 0.87)
- Motion Guided Spatial Attention for Video Captioning (AAAI 2019 · similarity 0.87)
- Re-thinking Temporal Search for Long-Form Video Understanding (CVPR 2025 · similarity 0.86)
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.86)
- Dual Prototype Attention for Unsupervised Video Object Segmentation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
