Research map / Architectures and efficiency / Attention and transformers
EMOv2: Pushing 5M Vision Model Frontier
Jiangning Zhang, Teng Hu, Haoyang He, Zhucun Xue, Yabiao Wang, Chengjie Wang, Yong Liu, Xiangtai Li, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SEW-MHSA spanning attention to the block (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 73.5 -> 77.7 | Improved |
| add DW-Conv to the block (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 73.5 -> 78.1 | Improved |
| use both SEW-MHSA and DW-Conv (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 73.5 -> 79.4 | Improved |
| use spanning attention rather than neighborhood attention (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 78.8 -> 79.4 | Improved |
| use spanning attention rather than remote attention (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 79.0 -> 79.4 | Improved |
| add attention at all stages (S-1234) instead of stage 3-4 (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 79.4 -> 79.1 | Hurt |
| run DW-Conv and attention in cascade rather than parallel (ablation) | EMOv1-5M · ImageNet-1K val | Top-1: 78.1 -> 78.3 | Improved |
| switch order to DW-Conv first (ablation) | EMOv1-5M / EMOv2-5M · ImageNet-1K val | Top-1: -0.6 (EMOv1), -0.7 (EMOv2) | Hurt |
| use kernel size 5 instead of 1 (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 78.6 -> 79.4 | Improved |
| replace DW-Conv with DCNv2 (ablation) | EMOv2-5M · ImageNet-1K val | Top-1: 79.4 -> 78.5 | Hurt |
| add iRMB to DeiT-Tiny | DeiT-Tiny · ImageNet-1K val | Top-1: 72.2 -> 74.3 | Improved |
| add i2RMB to DeiT-Tiny | DeiT-Tiny · ImageNet-1K val | Top-1: 72.2 -> 75.0 | Improved |
Closest papers by meaning
- Rethinking Mobile Block for Efficient Attention-based Models (ICCV 2023 · similarity 0.93)
- MobileMamba: Lightweight Multi-Receptive Visual Mamba Network (CVPR 2025 · similarity 0.85)
- EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba (AAAI 2025 · similarity 0.85)
- UNEM: UNrolled Generalized EM for Transductive Few-Shot Learning (CVPR 2025 · similarity 0.84)
- AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generation (NeurIPS 2024 · similarity 0.84)
- EMR-Merging: Tuning-Free High-Performance Model Merging (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
