Research map / Video / Video segmentation and tracking
Masked Autoencoders As Spatiotemporal Learners
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| MAE pre-training vs training from scratch | ViT-L · Kinetics-400 | top-1 accuracy: 71.4 -> 84.4 | Helped |
| MAE pre-training instead of supervised pre-training on K400 | ViT-L · SSv2 | top-1 accuracy: 55.7 -> 72.1 | Helped |
| pre-train on videos (K400) instead of IN1K images | MAE ViT-L · AVA | mAP: 26.3 -> 31.1 | Helped |
| drop mask tokens from encoder (sparse encoder) (ablation) | MAE · Kinetics-400 | wall-clock training time: 4.1x | Helped |
| drop mask tokens from encoder (ablation) | MAE · Kinetics-400 | top-1 accuracy: 84.3 -> 84.4 | No clear effect |
| spacetime-agnostic random masking vs time-only masking (ablation) | MAE · Kinetics-400 | top-1 accuracy: 79.1 -> 84.4 | Helped |
| spacetime-agnostic random masking vs space-only masking (ablation) | MAE · Kinetics-400 | top-1 accuracy: 83.5 -> 84.4 | Helped |
| per-patch normalized pixel target instead of raw pixels (ablation) | MAE · Kinetics-400 | top-1 accuracy: 83.8 -> 84.4 | Helped |
| add color jittering to augmentation (ablation) | MAE · Kinetics-400 | top-1 accuracy: 84.4 -> 83.8 | Hurt |
| repeated sampling 4x (ablation) | MAE · Kinetics-400 | speed: 1.0x -> 3.0x | Helped |
| widen decoder from 128-d to 512-d (ablation) | MAE · Kinetics-400 | top-1 accuracy: 80.8 -> 84.4 | Helped |
| deepen decoder from 1 to 4 blocks (ablation) | MAE · Kinetics-400 | top-1 accuracy: 83.2 -> 84.4 | Helped |
Closest papers by meaning
- Extending Video Masked Autoencoders to 128 frames (NeurIPS 2024 · similarity 0.90)
- EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokens (ICML 2024 · similarity 0.90)
- Self-Guided Masked Autoencoder (NeurIPS 2024 · similarity 0.89)
- R-MAE: Regions Meet Masked Autoencoders (ICLR 2024 · similarity 0.88)
- Understanding and Enhancing Mask-Based Pretraining towards Universal Representations (NeurIPS 2025 · similarity 0.88)
- The Dynamic Duo of Collaborative Masking and Target for Advanced Masked Autoencoder Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.