Research map / Architectures and efficiency / Attention and transformers
Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add feature reuse and relationship reuse (ablation) | DVT (T2T-ViT-12) 3rd exit · ImageNet val | top-1 accuracy: 76.74% -> 80.00% | Helped |
| add feature and relationship reuse (ablation) | DVT (T2T-ViT-12) 1st exit · ImageNet val | top-1 accuracy: 70.33% -> 69.04% | Hurt |
| add feature reuse only (ablation) | DVT (T2T-ViT-12) 3rd exit · ImageNet val | top-1 accuracy: 76.74% -> 79.21% | Helped |
| add relationship reuse only (ablation) | DVT (T2T-ViT-12) 3rd exit · ImageNet val | top-1 accuracy: 76.74% -> 78.86% | Helped |
| confidence-based exit instead of random exit (ablation) | DVT (T2T-ViT-12) · ImageNet val, 1.00G | top-1 accuracy: 71.66% -> 76.22% | Helped |
| confidence-based exit instead of entropy-based exit (ablation) | DVT (T2T-ViT-12) · ImageNet val, 1.00G | top-1 accuracy: 75.21% -> 76.22% | Helped |
| token-adaptive cascade (DVT) instead of layer-adaptive depth (ablation) | T2T-ViT-14 · ImageNet val, 2.00G | top-1 accuracy: 76.09% -> 79.24% | Helped |
| naive upsampling instead of proposed relationship upsample (ablation) | DVT relationship reuse · ImageNet val | top-1 accuracy | Hurt |
| use DVT instead of DeiT-small | DeiT-small · ImageNet val | top-1 accuracy: 79.80% -> 81.67% | Helped |
| use DVT instead of multi-exit nets (MSDNet, RANet) at 1.00G | DVT (T2T-ViT-12) · ImageNet val | top-1 accuracy: 72.24% -> 76.22% vs RANet | Helped |
Closest papers by meaning
- DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification (NeurIPS 2021 · similarity 0.92)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021 · similarity 0.91)
- An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025 · similarity 0.90)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.89)
- Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers (CVPR 2024 · similarity 0.88)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.