Research map / Architectures and efficiency / Attention and transformers
EdgeViTs: Competing Light-Weight CNNs on Mobile Devices with Vision Transformers
Junting Pan, Adrian Bulat, Fuwen Tan, Xiatian Zhu, Lukasz Dudziak, Hongsheng Li, Georgios Tzimiropoulos, Brais Martinez
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add local aggregation (with global sparse attention) rather than a single PVT-style attention block (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | top-1 accuracy (%): 72.7 -> 74.4 | Improved |
| use depthwise-conv local aggregation instead of locally-grouped self-attention (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | top-1 accuracy (%): 74.0 -> 74.4 | Improved |
| use depthwise-conv local aggregation instead of locally-grouped self-attention (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | CPU latency (ms): 36.1ms -> 32.8ms | Improved |
| sample delegate tokens by center instead of max (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | top-1 accuracy (%): 74.3 -> 74.4 | No clear effect |
| sample delegate tokens by center instead of avg (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | CPU latency (ms): 34.5ms -> 32.8ms | Improved |
| sample delegate tokens by avg instead of center (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | top-1 accuracy (%): 74.4 -> 74.5 | No clear effect |
| add local propagation (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | top-1 accuracy (%): 73.9 -> 74.4 | Improved |
| add local propagation (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | CPU latency (ms): 32.4ms -> 32.8ms | Hurt |
| use transposed convolution instead of bilinear upsampling for local propagation (ablation) | EdgeViT-XXS · ImageNet-1K val (ablation, ~0.5 GFLOPs) | top-1 accuracy (%): 74.1 -> 74.4 | Improved |
| EdgeViT-XXS instead of PVT-v2-B0 as backbone | EdgeViT-XXS · ADE20K val (Semantic FPN) | mIoU (%): 37.2 -> 39.7 | Improved |
| EdgeViT-S instead of PVT-v2-B1 as backbone | EdgeViT-S · ADE20K val (Semantic FPN) | mIoU (%): 42.5 -> 45.9 | Improved |
| EdgeViT-XXS instead of PVT-v2-B0 as RetinaNet backbone | EdgeViT-XXS · COCO val2017 (RetinaNet 1x) | AP: 37.2 -> 38.7 | Improved |
Closest papers by meaning
- RepViT: Revisiting Mobile CNN From ViT Perspective (CVPR 2024 · similarity 0.93)
- FasterViT: Fast Vision Transformers with Hierarchical Attention (ICLR 2024 · similarity 0.91)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.89)
- GradQ-ViT: Robust and Efficient Gradient Quantization for Vision Transformers (AAAI 2025 · similarity 0.89)
- SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design (CVPR 2024 · similarity 0.89)
- DeepCompress-ViT: Rethinking Model Compression to Enhance Efficiency of Vision Transformers at the Edge (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
