Research map / Video / Video-language understanding
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning
Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, Hongsheng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 3D depth-wise conv with temporal span (3x1x1) inside adapter vs 1x1x1 (ablation) | ST-Adapter · SSv2 val | top-1 accuracy: 46.2 -> 66.3 | Improved |
| use spatial-only kernel (1x3x3) instead of temporal kernel (ablation) | ST-Adapter · SSv2 val | top-1 accuracy: 46.2 -> 46.2 | No clear effect |
| kernel 3x3x3 vs 3x1x1 (ablation) | ST-Adapter · K400 val | top-1 accuracy: 82.0 -> 82.0 | No clear effect |
| add adapters to deeper blocks (9-12) instead of shallow (1-4) (ablation) | ST-Adapter · SSv2 val | top-1 accuracy: 45.9 -> 62.8 | Improved |
| add second ST-Adapter per block (before and after MHSA) (ablation) | ST-Adapter · SSv2 val | top-1 accuracy: 65.6 -> 67.0 | Improved |
| add second ST-Adapter per block (before and after MHSA) (ablation) | ST-Adapter · K400 val | top-1 accuracy: 82.0 -> 82.0 | No clear effect |
| increase bottleneck width from 64 to 384 (ablation) | ST-Adapter · K400 val | top-1 accuracy: 81.4 -> 82.0 | Improved |
| add ST-Adapter to frozen CLIP ViT-B (vs full fine-tuning) | CLIP ViT-B · SSv2 val | top-1 accuracy: 44.0 -> 66.3 | Improved |
| add ST-Adapter to frozen CLIP ViT-B/16 | CLIP ViT-B/16 · EK100 val | verb top-1: 54.8 -> 67.6 | Improved |
| replace ST-Adapter with full fine-tuning on ImageNet-21K SSv2 | ViT-B IN21K · SSv2 val | top-1 accuracy: 64.4 -> 62.8 | Hurt |
| use ST-Adapter on Swin-B instead of full fine-tuning | Swin-B · K400 val | top-1 accuracy: 81.5 -> 77.1 | Hurt |
| use ST-Adapter instead of standard Adapter (ablation) | CLIP ViT-B/16 · SSv2 val | top-1 accuracy: 46.2 -> 66.3 | Improved |
Later papers that compared with this method
6 later accepted papers report ST-Adapter as a baseline in their results tables; 6 of them report a better number on most of the results they share with it.
- Frame2Freq: Spectral Adapters for Fine-Grained Video Understanding (CVPR 2026 · better on 5 of 5 shared results)
- D2ST-Adapter: Disentangled-and-Deformable Spatio-Temporal Adapter for Few-shot Action Recognition (ICCV 2025 · better on 4 of 4 shared results)
- ZeroI2V: Zero-Cost Adaptation of Pre-Trained Transformers from Image to Video (ECCV 2024 · better on 2 of 2 shared results)
- Building an Open-Vocabulary Video CLIP Model With Better Architectures, Optimization and Data (TPAMI 2024 · better on 1 of 1 shared result)
- Dual-Path Adaptation From Image to Video Transformers (CVPR 2023 · better on 2 of 2 shared results)
- Open-VCLIP: Transforming CLIP to an Open-vocabulary Video Model via Interpolated Weight Optimization (ICML 2023 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Efficient Transfer Learning for Video-language Foundation Models (CVPR 2025 · similarity 0.89)
- VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding (AAAI 2024 · similarity 0.86)
- MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval (CVPR 2024 · similarity 0.86)
- Modality-Collaborative Test-Time Adaptation for Action Recognition (CVPR 2024 · similarity 0.85)
- MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition (ICML 2025 · similarity 0.85)
- A Multimodal, Multi-Task Adapting Framework for Video Action Recognition (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
