Research map / Architectures and efficiency / Attention and transformers
Revealing the Dark Secrets of Masked Image Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MIM rather than supervised ImageNet pretraining | SwinV2-B · COCO val pose estimation | AP: 75.2 -> 77.6 | Helped |
| Use MIM rather than supervised ImageNet pretraining | SwinV2-B · NYUv2 depth estimation | RMSE: 0.352 -> 0.304 | Helped |
| Use MIM rather than supervised ImageNet pretraining | SwinV2-B · Food101 | accuracy: 93.2 -> 94.2 | Helped |
| Use MIM rather than supervised ImageNet pretraining | SwinV2-B · CIFAR-10 | accuracy: 99.1 -> 99.2 | No clear effect |
| Use MIM rather than supervised ImageNet pretraining | SwinV2-B · COCO object detection | box AP: 51.9 -> 52.9 | Helped |
| Use MIM rather than supervised ImageNet pretraining | SwinV2-B · ADE-20K semantic segmentation with UperNet | mIoU: 50.9 -> 49.3 | Hurt |
| Randomly sample pretrained layer weights instead of preserving their order (ablation) | 1K-MIM · COCO pose estimation | AP: 75.5 -> 75.2 (-0.3) | Hurt |
| Randomly sample pretrained layer weights instead of preserving their order (ablation) | 1K-MIM · NYUv2 depth estimation | RMSE: 0.382 -> 0.434 (-0.052) | Hurt |
| Randomly sample pretrained layer weights instead of preserving their order (ablation) | 1K-SUP · COCO pose estimation | AP: 75.8 -> 74.9 (-0.9) | Hurt |
| Randomly sample pretrained layer weights instead of preserving their order (ablation) | 1K-SUP · NYUv2 depth estimation | RMSE: 0.376 -> 0.443 (-0.067) | Hurt |
| Drop two to eight final layers during fine-tuning (ablation) | supervised pre-trained SwinV2-B · COCO val2017 pose estimation | AP | Helped |
| Drop final layers during fine-tuning (ablation) | supervised pre-trained SwinV2-B · NYUv2 depth estimation | RMSE | No clear effect |
Closest papers by meaning
- On Data Scaling in Masked Image Modeling (CVPR 2023 · similarity 0.93)
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.91)
- Morphing Tokens Draw Strong Masked Image Models (ICLR 2025 · similarity 0.90)
- Pre-training with Random Orthogonal Projection Image Modeling (ICLR 2024 · similarity 0.89)
- On the Role of Discrete Tokenization in Visual Representation Learning (ICLR 2024 · similarity 0.87)
- Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.