Research map / Architectures and efficiency / Attention and transformers
Good Helper Is around You: Attention-Driven Masked Image Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use attention-driven masking and throwing instead of random masking | MAE · CIFAR-10 | linear probing Top-1 accuracy: 85.2 -> 88.1 | Helped |
| use attention-driven masking and throwing instead of random masking | MAE · CIFAR-100 | linear probing Top-1 accuracy: 65.2 -> 68.7 | Helped |
| use attention-driven masking and throwing instead of random masking | MAE · Tiny ImageNet | linear probing Top-1 accuracy: 55.2 -> 59.6 | Helped |
| use attention-driven masking and throwing instead of random masking | MAE · STL-10 | linear probing Top-1 accuracy: 80.9 -> 86.8 | Helped |
| use attention-driven masking and throwing instead of random masking | MAE · ImageNet-1K | linear probing Top-1 accuracy: 56.6 -> 61.7 | Helped |
| use AMT instead of random masking | MAE · fine-tuning across CIFAR-10/100, Tiny ImageNet, STL-10, ImageNet-1K | Top-1 accuracy: 96.5 -> 97.5; 87.4 -> 87.8; 76.5 -> 77.8; 96.5 -> 97.5; 82.6 -> 82.8 | Mixed |
| use AMT instead of random masking | SimMIM · fine-tuning across CIFAR-10/100, Tiny ImageNet, STL-10, ImageNet-1K | Top-1 accuracy: 95.0 -> 97.7; 80.3 -> 86.1; 74.0 -> 75.8; 92.3 -> 96.5; 81.5 -> 80.7 | Mixed |
| use attention-driven masking and throwing | MAE · LVIS | APbbox: 25.0 -> 26.9 | Helped |
| use attention-driven masking and throwing | MAE · COCO | APbbox: 40.1 -> 42.8 | Helped |
| use attention-driven masking and throwing | MAE · COCO | APmask: 36.6 -> 36.6 | No clear effect |
| add attention-driven masking without throwing (ablation) | MAE · Tiny ImageNet | linear probing Top-1 accuracy: 52.3 -> 50.1 | Hurt |
| add random throwing of 40% tokens (ablation) | MAE · Tiny ImageNet | linear probing Top-1 accuracy: 52.3 -> 50.8 | Hurt |
Closest papers by meaning
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.91)
- Masked Image Modeling With Local Multi-Scale Reconstruction (CVPR 2023 · similarity 0.90)
- Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representation (AAAI 2025 · similarity 0.89)
- Learning Mask Invariant Mutual Information for Masked Image Modeling (ICLR 2025 · similarity 0.88)
- Self-Guided Masked Autoencoder (NeurIPS 2024 · similarity 0.88)
- Morphing Tokens Draw Strong Masked Image Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.