Research map / Architectures and efficiency / Attention and transformers
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adaptively select patches, heads, and blocks per image | T2T-ViT-19 · ImageNet validation | Top-1 accuracy (%): 81.9 -> 81.1 | Hurt |
| Adaptively select patches, heads, and blocks per image | T2T-ViT-19 · ImageNet validation | FLOPs (G): 8.5 -> 3.9 | Helped |
| Replace learned patch policies with random policies (ablation) | AdaViT · ImageNet validation | Top-1 accuracy (%): 81.1 -> 49.2 | Hurt |
| Replace learned head policies with random policies (ablation) | AdaViT · ImageNet validation | Top-1 accuracy (%): 81.1 -> 57.4 | Hurt |
| Replace learned block policies with random policies (ablation) | AdaViT · ImageNet validation | Top-1 accuracy (%): 81.1 -> 64.7 | Hurt |
| Use partial instead of full attention-head deactivation at 50% heads (ablation) | AdaViT · ImageNet validation | Top-1 accuracy (%): 80.3 -> 81.7 | Helped |
| Use partial instead of full attention-head deactivation at 50% heads (ablation) | AdaViT · ImageNet validation | FLOPs (G): 5.1 -> 6.9 | Hurt |
| Keep 60% rather than 50% of heads with full deactivation (ablation) | AdaViT · ImageNet validation | Top-1 accuracy (%): 80.3 -> 80.8 | Helped |
| Keep 60% rather than 50% of heads with full deactivation (ablation) | AdaViT · ImageNet validation | FLOPs (G): 5.1 -> 5.8 | Hurt |
| Keep 70% rather than 50% of heads with full deactivation (ablation) | AdaViT · ImageNet validation | Top-1 accuracy (%): 80.3 -> 81.1 | Helped |
| Keep 70% rather than 50% of heads with full deactivation (ablation) | AdaViT · ImageNet validation | FLOPs (G): 5.1 -> 6.6 | Hurt |
Closest papers by meaning
- A-ViT: Adaptive Tokens for Efficient Vision Transformer (CVPR 2022 · similarity 0.91)
- Skip-Attention: Improving Vision Transformers by Paying Less Attention (ICLR 2024 · similarity 0.90)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.89)
- An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025 · similarity 0.89)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.89)
- Bi-ViT: Pushing the Limit of Vision Transformer Quantization (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.