Research map / Architectures and efficiency / Attention and transformers
EnViT: Enhancing the Performance of Early-Exit Vision Transformers via Exit-Aware Structured Dropout-Enabled Self-Distillation
Yonghao Dong, Qiang He, Penghong Rui, Zhenzhe Zheng, Zhao Li, Feifei Chen, Hai Jin, Yun Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add exit-aware structured dropout (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 exit s2 | accuracy: 75.61 -> 76.23 | Improved |
| add exit-aware structured dropout (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 final exit s6 | accuracy: 89.40 -> 89.88 | Improved |
| add self-distillation without structured dropout (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 exit s2 | accuracy: 75.61 -> 78.05 | Improved |
| add self-distillation without structured dropout (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 final exit s6 | accuracy: 89.40 -> 87.73 | Hurt |
| add self-distillation without structured dropout (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 exit s5 | accuracy: 88.85 -> 87.68 | Hurt |
| add self-distillation without structured dropout (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 exit s4 | accuracy: 87.21 -> 86.51 | Hurt |
| combine structured dropout and self-distillation (full EnViT) (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 exit s1 | accuracy: 60.48 -> 61.94 | Improved |
| combine structured dropout and self-distillation (full EnViT) (ablation) | early-exit ViT-B/16 baseline · CIFAR-100 exit s2 | accuracy: 75.61 -> 79.02 | Improved |
| add structured dropout on top of self-distillation (ablation) | EnViT-KD · CIFAR-100 final exit s6 | accuracy: 87.73 -> 89.94 | Improved |
| add self-distillation on top of structured dropout (ablation) | EnViT-SD · CIFAR-100 exit s2 | accuracy: 76.23 -> 79.02 | Improved |
| replace LGViT with EnViT | LGViT · CIFAR-100 | top-1 accuracy: 88.14% -> 89.23% | Improved |
| replace ViT-EE with EnViT | ViT-EE · ImageNet-1k | top-1 accuracy: 77.57% -> 80.45% | Improved |
Closest papers by meaning
- TinyViT: Fast Pretraining Distillation for Small Vision Transformers (ECCV 2022 · similarity 0.89)
- EdgeViTs: Competing Light-Weight CNNs on Mobile Devices with Vision Transformers (ECCV 2022 · similarity 0.88)
- You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model (CVPR 2023 · similarity 0.88)
- AdaViT: Adaptive Vision Transformers for Efficient Image Recognition (CVPR 2022 · similarity 0.88)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.87)
- Amortized-Precision Quantization for Early-Exit Vision Transformers (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
