Research map / Video / Action recognition
AIM: Adapting Image Models for Efficient Video Action Recognition
Taojiannan Yang, Yi Zhu, Yusheng Xie, Aston Zhang, Chen Chen, Mu Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add spatial adapters to the frozen space-only model (ablation) | ViT-B/16 IN-21K on Something-Something-v2 · Something-Something-v2 | Top-1: 15.1 -> 36.7 | Improved |
| Add temporal adaptation to spatial adaptation (ablation) | ViT-B/16 IN-21K on Something-Something-v2 · Something-Something-v2 | Top-1: 36.7 -> 61.2 | Improved |
| Add joint adaptation to spatial and temporal adaptation (ablation) | ViT-B/16 IN-21K on Something-Something-v2 · Something-Something-v2 | Top-1: 61.2 -> 62.0 | Improved |
| Replace IN-21K image pretraining with CLIP pretraining (ablation) | AIM ViT-B/16 on Something-Something-v2 · Something-Something-v2 | Top-1: 62.0 -> 66.4 | Improved |
| Fully finetune the space-only model instead of freezing it (ablation) | ViT-B/16 IN-21K on Something-Something-v2 · Something-Something-v2 | Top-1: 15.1 -> 36.2 | Improved |
| Use AIM rather than full finetuning | ViT-B/16 IN-21K on Something-Something-v2 · Something-Something-v2 | Top-1: 59.5 -> 62.0 | Improved |
| Use AIM rather than EVL | CLIP ViT-L/14 on Something-Something-v2 · Something-Something-v2 | Top-1: 66.7 -> 70.6 | Improved |
| Increase the number of input frames (ablation) | AIM ViT-B/16 on Kinetics-400 · Kinetics-400 | Top-1: 83.9 -> 84.5 -> 84.7 | Improved |
| Use AIM rather than TimeSformer | ViT-B IN-21K on Kinetics-400 · Kinetics-400 | Top-1: 78.5 -> 78.8 | Improved |
| Use AIM rather than VideoSwin-B | Swin-B IN-21K on Kinetics-400 · Kinetics-400 | Top-1: 82.7 -> 82.1 | Hurt |
| Place adapters in the bottom 6 blocks rather than deeper blocks (ablation) | AIM · Kinetics-400 | Top-1 | Hurt |
| Place adapters in the top 6 blocks instead of all blocks (ablation) | AIM · Kinetics-400 | Top-1 | No clear effect |
Closest papers by meaning
- ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning (NeurIPS 2022 · similarity 0.90)
- Efficient Transfer Learning for Video-language Foundation Models (CVPR 2025 · similarity 0.89)
- Align Before Adapt: Leveraging Entity-to-Region Alignments for Generalizable Video Action Recognition (CVPR 2024 · similarity 0.86)
- Adapting Short-Term Transformers for Action Detection in Untrimmed Videos (CVPR 2024 · similarity 0.86)
- Time- Memory- and Parameter-Efficient Visual Adaptation (CVPR 2024 · similarity 0.86)
- Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation? (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
