AI Paper Atlas

Research map / Video / Video-language understanding

Text-Guided Video Masked Autoencoder

David Fan, Jue Wang, Shuai Liao, Zhikang Zhang, Vimal Bhat, Xinyu Li

ECCV 2024 · Video › Video-language understanding · approach: Video-language pretraining · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add masked video-text contrastive loss to MAE (ablation)tube masking, SSv2 · SSv2linear probe top-1: 20.8 -> 33.3Improved
add masked video-text contrastive loss to MAE (ablation)text-guided masking, SSv2 · SSv2linear probe top-1: 26.2 -> 33.4Improved
add masked video-text contrastive loss to MAE (ablation)MGM, SSv2 · SSv2finetune top-1: 67.3 -> 67.0No clear effect
contrastive loss alone without MAE (ablation)tube masking, SSv2 · SSv2finetune top-1: 64.9 -> 44.9Hurt
add contrastive loss to text-guided masking on K400-pretrained linear probe (ablation)text-guided masking · UCF101linear probe top-1: 67.7 -> 87.1Improved
add contrastive loss to text-guided masking on Epic-Kitchens (ablation)text-guided masking · Epic-Kitchens 55finetune top-1: 34.7 -> 37.2Improved
text-guided masking versus tube masking in pure MAE on SSv2MAE · SSv2finetune top-1: 66.6 -> 67.1Improved
text-guided masking versus tube masking in pure MAE on K400 linear probeMAE · K400linear probe top-1: 38.1 -> 33.8Hurt
raise mask ratio from 0.6 to 0.75 (ablation)text-guided masking · SSv2finetune top-1: 67.5 -> 66.4Hurt
mask top-K instead of bottom-K text-similar patches (ablation)text-guided masking · SSv2finetune top-1: 67.2 -> 67.5Improved
use 3 captions per video instead of 1 (ablation)text-guided masking · SSv2finetune top-1: 66.5 -> 67.5Improved
replace BLIP captions with vision-free GPT3.5 captions (ablation)text-guided masking + contrastive · SSv2finetune / linear probe: FT 67.5 -> 66.2; LP 33.4 -> 54.0Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy