Research map / Architectures and efficiency / Attention and transformers
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| dynamic token sparsification at rho=0.7 | DeiT-S · ImageNet-1k | throughput: 1337.7 -> 2062.1 | Helped |
| dynamic token sparsification at rho=0.7 | DeiT-S · ImageNet-1k | top-1 accuracy: 79.8 -> 79.3 | Hurt |
| dynamic sparsification instead of static sparsification (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy: 73.4 -> 79.3 | Helped |
| dynamic sparsification instead of structural downsampling (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy: 78.2 -> 79.3 | Helped |
| learned prediction module instead of random token removal (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy: 77.5 -> 79.3 | Helped |
| learned prediction module instead of class-attention removal (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy: 78.1 -> 79.3 | Helped |
| progressive three-stage instead of single-stage sparsification (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy: 77.4 -> 79.3 | Helped |
| add distillation and KL losses (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy | Helped |
| keeping ratio below 0.7 (ablation) | DynamicViT · ImageNet-1k | top-1 accuracy | Hurt |
Closest papers by meaning
- Frequency-Aware Token Reduction for Efficient Vision Transformer (NeurIPS 2025 · similarity 0.89)
- Multi-criteria Token Fusion with One-step-ahead Attention for Efficient Vision Transformers (CVPR 2024 · similarity 0.89)
- Not All Tokens Matter All The Time: Dynamic Token Aggregation Towards Efficient Detection Transformers (ICML 2025 · similarity 0.89)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.89)
- Token Cropr: Faster ViTs for Quite a Few Tasks (CVPR 2025 · similarity 0.88)
- Exploring Token Pruning in Vision State Space Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.