Research map / Video / Video-language understanding
Text-Guided Video Masked Autoencoder
David Fan, Jue Wang, Shuai Liao, Zhikang Zhang, Vimal Bhat, Xinyu Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add masked video-text contrastive loss to MAE (ablation) | tube masking, SSv2 · SSv2 | linear probe top-1: 20.8 -> 33.3 | Improved |
| add masked video-text contrastive loss to MAE (ablation) | text-guided masking, SSv2 · SSv2 | linear probe top-1: 26.2 -> 33.4 | Improved |
| add masked video-text contrastive loss to MAE (ablation) | MGM, SSv2 · SSv2 | finetune top-1: 67.3 -> 67.0 | No clear effect |
| contrastive loss alone without MAE (ablation) | tube masking, SSv2 · SSv2 | finetune top-1: 64.9 -> 44.9 | Hurt |
| add contrastive loss to text-guided masking on K400-pretrained linear probe (ablation) | text-guided masking · UCF101 | linear probe top-1: 67.7 -> 87.1 | Improved |
| add contrastive loss to text-guided masking on Epic-Kitchens (ablation) | text-guided masking · Epic-Kitchens 55 | finetune top-1: 34.7 -> 37.2 | Improved |
| text-guided masking versus tube masking in pure MAE on SSv2 | MAE · SSv2 | finetune top-1: 66.6 -> 67.1 | Improved |
| text-guided masking versus tube masking in pure MAE on K400 linear probe | MAE · K400 | linear probe top-1: 38.1 -> 33.8 | Hurt |
| raise mask ratio from 0.6 to 0.75 (ablation) | text-guided masking · SSv2 | finetune top-1: 67.5 -> 66.4 | Hurt |
| mask top-K instead of bottom-K text-similar patches (ablation) | text-guided masking · SSv2 | finetune top-1: 67.2 -> 67.5 | Improved |
| use 3 captions per video instead of 1 (ablation) | text-guided masking · SSv2 | finetune top-1: 66.5 -> 67.5 | Improved |
| replace BLIP captions with vision-free GPT3.5 captions (ablation) | text-guided masking + contrastive · SSv2 | finetune / linear probe: FT 67.5 -> 66.2; LP 33.4 -> 54.0 | Mixed |
Closest papers by meaning
- Extending Video Masked Autoencoders to 128 frames (NeurIPS 2024 · similarity 0.90)
- Motion-Guided Masking for Spatiotemporal Representation Learning (ICCV 2023 · similarity 0.89)
- Video Language Model Pretraining with Spatio-temporal Masking (CVPR 2025 · similarity 0.87)
- VideoMAC: Video Masked Autoencoders Meet ConvNets (CVPR 2024 · similarity 0.87)
- A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames (CVPR 2024 · similarity 0.86)
- Learning Mask Invariant Mutual Information for Masked Image Modeling (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
