AI Paper Atlas

Research map / Video / Video segmentation and tracking

ZeroI2V: Zero-Cost Adaptation of Pre-Trained Transformers from Image to Video

Xinhao Li, Yuhan Zhu, Limin Wang

ECCV 2024 · Video › Video segmentation and tracking · approach: Self-supervised video representation learning · 20 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
relocate heads (STDHA) instead of temporal channel shift on KV (ablation)ViT-B/16 temporal modeling · SSv2 valtop-1 accuracy: 64.6 -> 66.0Improved
apply head relocation to KV only rather than QKV (ablation)STDHA · SSv2 valtop-1 accuracy: 64.8 -> 66.0Improved
add Linear Adapter with full adaptation instead of full fine-tuning (ablation)CLIP ViT-B/16 · SSv2 valtop-1 accuracy: 63.2 -> 66.3Improved
tune only the attention weights (temporal head) instead of full adaptation (ablation)Linear Adapter · SSv2 valtop-1 accuracy: 63.2 -> 59.6 vs full fine-tuningHurt
densely place adapters on all weights (full adaptation) (ablation)linear and non-linear adapters · SSv2 valtop-1 accuracy: Linear Adapter 64.4 -> 66.0Improved
use serial Linear Adapter instead of parallel LoRA (ablation)image-to-video adaptation · SSv2 valtop-1 accuracy: LoRA 65.5 -> Linear Adapter 66.2Improved
replace linear adapter with non-linear GELU adapter (ablation)Linear Adapter · SSv2 valtop-1 accuracy: 66.0 vs 66.1No clear effect
raise adapter bottleneck ratio from 0.25 to 0.5 (ablation)Linear Adapter · SSv2 valtop-1 accuracy: 66.0 -> 65.8No clear effect
use more temporal heads (more than 1/6 to 1/4 of heads) (ablation)STDHA ViT-B · SSv2 valtop-1 accuracy: 66.0 -> 65.6Hurt
enlarge temporal receptive field beyond 3 for 8 frames (ablation)STDHA · SSv2 val, 8 framestop-1 accuracy: 66.0 -> 65.5No clear effect
enlarge temporal receptive field for 32 frames (ablation)STDHA · SSv2 val, 32 framestop-1 accuracy: 67.3 -> 68.6Improved
use ZeroI2V instead of full fine-tuning of CLIP ViT-L/14 on SSv2CLIP ViT-L/14 · SSv2 valtop-1 accuracy: 48.7 -> 70.1Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy