Research map / Detection and segmentation / CNN features
Bootstrapped Masked Autoencoders for Vision BERT Pretraining
Xiaoyi Dong, Jianmin Bao, Ting Zhang, Dongdong Chen, Weiming Zhang, Lu Yuan, Dong Chen, Fang Wen, Nenghai Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add bootstrapped feature prediction (momentum-encoder target) (ablation) | MAE, 300 epochs · ImageNet-1K val | top-1 accuracy: 83.2 -> 83.6 | Improved |
| add feature injection context for the regressor (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.6 -> 83.9 | Improved |
| add feature injection context for the predictor as well (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.9 -> 84.0 | Improved |
| use block-wise instead of random masking with feature-level target (ablation) | MAE 300 epochs · ImageNet-1K val | top-1 accuracy: 83.1 -> 83.6 | Improved |
| use block-wise instead of random masking with pixel-level target (ablation) | MAE 300 epochs · ImageNet-1K val | top-1 accuracy: 83.2 -> 82.8 | Hurt |
| increase regressor/predictor depth from 1 to 2 (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.4 -> 84.0 | Improved |
| increase depth from 2 to 8 (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 84.0 -> 84.0 | No clear effect |
| increase regressor dimension from 256 to 512 (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.8 -> 84.0 | Improved |
| feed 100% rather than 25% of the image to the momentum encoder (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.8 -> 84.0 | Improved |
| feed 100% rather than 25% of image to momentum encoder: training iteration time (ablation) | BootMAE · ImageNet-1K val | iteration time (s): 0.407 -> 0.660 | Hurt |
| use low-level instead of high-level context for pixel regressor (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.4 -> 84.0 | Improved |
| use low-level context for pixel regressor instead of middle-level (predictor high-level) (ablation) | BootMAE · ImageNet-1K val | top-1 accuracy: 83.7 -> 84.0 | Improved |
Closest papers by meaning
- Mc-BEiT: Multi-Choice Discretization for Image BERT Pre-training (ECCV 2022 · similarity 0.88)
- Understanding and Enhancing Mask-Based Pretraining towards Universal Representations (NeurIPS 2025 · similarity 0.87)
- Exploring Target Representations for Masked Autoencoders (ICLR 2024 · similarity 0.86)
- R-MAE: Regions Meet Masked Autoencoders (ICLR 2024 · similarity 0.86)
- Bootstrapping SparseFormers from Vision Foundation Models (CVPR 2024 · similarity 0.85)
- DDAE: Towards Deep Dynamic Vision BERT Pretraining (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
