AI Paper Atlas

Research map / Video / Video-language understanding

Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning

Zhiwu Qing, Shiwei Zhang, Ziyuan Huang, Yingya Zhang, Changxin Gao, Deli Zhao, Nong Sang

ICCV 2023 · Video › Video-language understanding · approach: Long video understanding · 41 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the temporal encoder to the spatial encoder alone (ablation)DiST · SSV2, ViT-B/16 8 framesTop-1 accuracy: 55.0 -> 63.2Improved
add integration branch to the spatial encoder alone (ablation)DiST · SSV2, ViT-B/16 8 framesTop-1 accuracy: 55.0 -> 65.9Improved
add the temporal encoder on top of the integration branch (ablation)DiST · SSV2, ViT-B/16Top-1 accuracy: 65.9 -> 68.7Improved
add bidirectional feature interactions between temporal encoder and integration branch (ablation)DiST · SSV2, ViT-B/16Top-1 accuracy: 67.5 -> 68.7Improved
use R(2+1)D as T-Block rather than TAdaConv, C3D or joint transformer (ablation)DiST temporal encoder · SSV2Top-1 accuracy: 67.8 -> 68.7Improved
increase temporal encoder frames gamma from 1 to 2 (ablation)DiST · SSV2Top-1 accuracy: 67.9 -> 68.7Improved
increase temporal encoder frames beyond gamma=4 (ablation)DiST · SSV2Top-1 accuracy: 69.1 (gamma=4) -> 68.5 (gamma=8)No clear effect
widen integration branch from alpha=1/8 to 1/2 (ablation)DiST · SSV2Top-1 accuracy: 62.6 -> 68.7Improved
widen integration branch from alpha=1/2 to 1 (ablation)DiST · SSV2Top-1 accuracy: 68.7 -> 68.7No clear effect
use nearest interpolation instead of deconvolution for upsampling (ablation)DiST feature interaction · SSV2Top-1 accuracy: 67.8 -> 68.7Improved
use low-level features from all ViT layers rather than only deep layers 9-12 (ablation)DiST · SSV2Top-1 accuracy: 66.0 -> 68.7Improved
use DiST disentangled design instead of EVL decoder with frozen CLIPfrozen CLIP ViT-B/16 · SSV2 32 framesTop-1 accuracy: 62.4 -> 70.9Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy