Research map / Label-efficient and robust learning / Self-supervised and contrastive
Beyond [cls]: Exploring the True Potential of Masked Image Modeling Representations
Marcin Przewięźlikowski, Randall Balestriero, Wojciech Jasiński, Marek Śmieja, Bartosz Zieliński
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace [cls] token with Selective Aggregation (AbMILP) of patch tokens | MAE ViT-B · ImageNet-1k linear probe | top-1 accuracy: 67.8 -> 71.6 | Improved |
| replace [cls] with Selective Aggregation on SimMIM | SimMIM ViT-B · ImageNet-1k linear probe | top-1 accuracy: 51.5 -> 62.8 | Improved |
| replace average pooling with Selective Aggregation on CAPI | CAPI ViT-L · ImageNet-1k linear probe | top-1 accuracy: 76.2 -> 82.4 | Improved |
| apply Selective Aggregation to JEA models (DINO, MoCo-v3) | DINO ViT-B · ImageNet-1k linear probe | top-1 accuracy: 76.6 -> 75.2 | No clear effect |
| apply Selective Aggregation to fine-tuned MAE | MAE ViT-B fine-tuned · ImageNet-1k linear probe | top-1 accuracy: 80.0 -> 79.1 | Hurt |
| also aggregate [cls] token with patches (ablation) | Selective Aggregation on MIMs · ImageNet-1k linear probe | top-1 accuracy: 71.6 -> 71.5 (MAE ViT-B) | No clear effect |
| apply Selective Aggregation for fine-grained classification | MAE ViT-B · CUB-200 | accuracy: 45.8 -> 65.9 | Improved |
| apply Selective Aggregation on BEIT-v2 | BEIT-v2 ViT-B · OxfordIIIPets | accuracy: 93.7 -> 93.4 | No clear effect |
| replace Tanh with ReLU in AbMILP MLP scorer (ablation) | AbMILP aggregation · ImageNet-1k linear probe | top-1 accuracy: 68.7 -> 71.7 | Improved |
| use deeper MLP scorer instead of linear scorer in AbMILP (ablation) | AbMILP aggregation · ImageNet-1k linear probe | top-1 accuracy: 71.6 -> 71.5 | No clear effect |
| use learned AbMILP instead of non-trainable attention-based aggregation (ablation) | MAE ViT-B · ImageNet-1k linear probe | top-1 accuracy: 65.2 -> 71.6 | Improved |
| use 12-head Attentive Probing instead of AbMILP (ablation) | MAE ViT-B · ImageNet-1k linear probe | top-1 accuracy: 71.6 -> 75.4 | Improved |
Closest papers by meaning
- Towards Latent Masked Image Modeling for Self-Supervised Visual Representation Learning (ECCV 2024 · similarity 0.93)
- Masked Image Modeling: A Survey (IJCV 2025 · similarity 0.93)
- Revealing the Dark Secrets of Masked Image Modeling (CVPR 2023 · similarity 0.93)
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.91)
- On the Role of Discrete Tokenization in Visual Representation Learning (ICLR 2024 · similarity 0.89)
- Morphing Tokens Draw Strong Masked Image Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
