Research map / Architectures and efficiency / Attention and transformers
Faster Vision Transformers with Adaptive Patches
Rohan Choudhury, JungEun Kim, Jinhyung Park, Eunho Yang, Laszlo A. Jeni, Kris Kitani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use zero-initialized connection instead of a residual connection (ablation) | APT · ImageNet-1k (with training) | accuracy: 87.52 -> 88.13 | Improved |
| use zero-initialized connection instead of a non-zero initialized connection (ablation) | APT · ImageNet-1k (with training) | accuracy: 87.81 -> 88.13 | Improved |
| use zero-initialized connection instead of a residual connection (ablation) | APT · ImageNet-1k (without training) | accuracy: 87.40 -> 87.98 | Improved |
| use adaptive patches instead of random token dropping (ablation) | ViT-B MAE 384/16 · ImageNet-1k full fine-tuning | accuracy: 83.4 -> 84.2 | Improved |
| use adaptive patches instead of resizing only (ablation) | ViT-L MAE 336/14 · ImageNet-1k full fine-tuning | accuracy: 85.9 -> 86.1 | Improved |
| use adaptive patches instead of resizing only (ablation) | EVA-02-L · ADE20K 512 | mIoU: 58.81 -> 59.70 | Improved |
| use adaptive patches instead of resizing only (ablation) | LLaVA-1.5-13B · VQAv2 | accuracy: 78.9 -> 79.4 | Improved |
| apply APT to ViT-B at 384/16 for full fine-tuning | ViT-B MAE · ImageNet-1k | training time: 11.6h -> 9.0h; +29% | Improved |
| apply APT to ViT-L at 448/14 | ViT-L MAE · ImageNet-1k | training time: 31.4h -> 16.9h; +86% | Improved |
| apply APT with no token reduction (tau = -1) (ablation) | ViT-B 224/16 · ImageNet-1k throughput | Img/s: 3310 -> 3090 | Hurt |
| apply APT with no token reduction (tau = -1) (ablation) | ViT-L 336/14 · ImageNet-1k throughput | Img/s: 395 -> 360 | Hurt |
| increase entropy threshold (ablation) | APT · ImageNet-1k | accuracy | Hurt |
Closest papers by meaning
- FlexiViT: One Model for All Patch Sizes (CVPR 2023 · similarity 0.92)
- Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers (NeurIPS 2026 · similarity 0.91)
- Slicing Vision Transformer for Flexible Inference (NeurIPS 2024 · similarity 0.88)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.88)
- Linearly Decomposing and Recomposing Vision Transformers for Diverse-Scale Models (NeurIPS 2024 · similarity 0.88)
- Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And More (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
