AI Paper Atlas

Research map / Video / Video segmentation and tracking

Motion-Guided Masking for Spatiotemporal Representation Learning

David Fan, Jue Wang, Shuai Liao, Yi Zhu, Vimal Bhat, Hector Santos-Villalobos, Rohith MV, Xinyu Li

ICCV 2023 · Video › Video segmentation and tracking · approach: Self-supervised video representation learning · 26 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace random tube masking with motion-guided 3D masking (ablation)Video MAE (ViT-B) · SSv2 (200 ep pretrain)top-1 accuracy: 65.0 -> 67.3Improved
make mask spatially dense (continuous) instead of sparse (ablation)MGM · SSv2 (200 ep pretrain)top-1 accuracy: 65.1 -> 67.3Improved
guide mask with real motion vectors instead of simulated random motion (ablation)Dense masking · SSv2 (200 ep pretrain)top-1 accuracy: 65.9 -> 67.3Improved
use static block mask without motion (ablation)Masking strategy · SSv2 (200 ep pretrain)top-1 accuracy: 67.3 -> 62.9Hurt
use motion vectors instead of optical flow as guidance (ablation)MGM · SSv2 (200 ep pretrain)top-1 accuracy: 66.0 -> 67.3Improved
add mask aspect ratio jitter (ablation)MGM · SSv2 (200 ep pretrain)top-1 accuracy: 67.1 -> 67.3No clear effect
pretrain on 80K-video Mini-Kinetics-200 instead of K400 (ablation)MGM · K400 (200 ep pretrain)top-1 accuracy: 78.6 -> 78.2Hurt
use ViT-B instead of ViT-S backbone (ablation)MGM · SSv2 (200 ep pretrain)top-1 accuracy: 61.1 -> 67.3Improved
lower the masking ratio to 0.75 for motion-guided masks (ablation)MGM · SSv2 (200 ep pretrain)top-1 accuracyImproved
replace VideoMAE random tube masking with MGM, 800 epochsVideoMAE · SSv2top-1 accuracy: 69.6 -> 70.6Improved
replace VideoMAE masking with MGM, 800 epochsVideoMAE · Kinetics-400top-1 accuracy: 80.0 -> 80.8Improved
MGM versus VideoMAE for linear probing on K400 pretrainingVideoMAE · UCF101 linear probetop-1 accuracy: 70.5 -> 77.5Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy