Research map / Architectures and efficiency / Attention and transformers
A-ViT: Adaptive Tokens for Efficient Vision Transformer
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Halt spatial tokens independently at different depths instead of halting every token together (ablation) | DeiT-Tiny with ACT-style halting · ImageNet1K validation | FLOPs: 1.0G -> 0.8G | Helped |
| Add a distributional prior on halting scores (ablation) | A-ViT-T · ImageNet1K validation | Top-1 accuracy: +6.4% | Helped |
| Use the distributional prior alone without ACT ponder loss (ablation) | A-ViT-T · ImageNet1K validation | Top-1 accuracy: more than 2% | Hurt |
| Use one embedding dimension for halting, set to zero, without fine-tuning (ablation) | DeiT-T/S · ImageNet1K | Top-1 accuracy: 0.08% ± 0.04%/0.04% ± 0.03% | Hurt |
| Add a two-layer auxiliary network for halting (ablation) | A-ViT-T · ImageNet1K | Top-1 accuracy: +0.06% | Helped |
| Add a two-layer auxiliary network for halting (ablation) | A-ViT-T · ImageNet1K | inference throughput: −12.6% | Hurt |
| Apply adaptive token halting | DeiT-Small · ImageNet1K | throughput: 0.8K imgs/s -> 1.1K imgs/s | Helped |
| Apply adaptive token halting | DeiT-Tiny · ImageNet1K | throughput: 2.1K imgs/s -> 3.4K imgs/s | Helped |
Closest papers by meaning
- Slicing Vision Transformer for Flexible Inference (NeurIPS 2024 · similarity 0.90)
- Bi-ViT: Pushing the Limit of Vision Transformer Quantization (AAAI 2024 · similarity 0.89)
- Skip-Attention: Improving Vision Transformers by Paying Less Attention (ICLR 2024 · similarity 0.89)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.88)
- Token Cropr: Faster ViTs for Quite a Few Tasks (CVPR 2025 · similarity 0.88)
- Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.