Research map / Architectures and efficiency / Attention and transformers
Mobile-Former: Bridging MobileNet and Transformer
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the parallel MobileNet–Transformer design with a bidirectional bridge instead of a Mobile-only network (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 74.2 -> 76.8 | Helped |
| Add dynamic ReLU to the Mobile branch, with parameters generated from the first global token (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: +1.0% | Helped |
| Increase global token count from 1 to 3 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.1 -> 77.6 | Helped |
| Increase global token count from 3 to 6 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.6 -> 77.8 | Helped |
| Increase global token count beyond 6 to 9 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.8 -> 77.7 | No clear effect |
| Increase token dimension from 64 to 192 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 76.8 -> 77.8 | Helped |
| Increase token dimension beyond 192 to 256 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.8 -> 77.8 | No clear effect |
| Increase token dimension from 192 to 320 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.8 -> 77.6 | Hurt |
| Remove the Former FFN (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.8 -> 77.5 | Hurt |
| Replace MHA with position-mixing MLP in Former and bridge (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.8 -> 77.3 | Hurt |
| Increase depthwise kernel size from 3×3 to 5×5 (ablation) | Mobile-Former-294M · ImageNet classification | Top-1 accuracy: 77.8 -> 77.9 | No clear effect |
| Replace ResNet-50 backbone with MF-508M in the DETR setup (ablation) | End-to-end detector · COCO val2017 | AP: 40.6 -> 39.4 | Hurt |
Closest papers by meaning
- IFORMER: INTEGRATING CONVNET AND TRANSFORMER FOR MOBILE APPLICATION (ICLR 2025 · similarity 0.89)
- RepViT: Revisiting Mobile CNN From ViT Perspective (CVPR 2024 · similarity 0.86)
- Mobile Attention: Mobile-Friendly Linear-Attention for Vision Transformers (ICML 2024 · similarity 0.86)
- CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction (CVPR 2025 · similarity 0.85)
- MobileInst: Video Instance Segmentation on the Mobile (AAAI 2024 · similarity 0.84)
- DECO: Unleashing the Potential of ConvNets for Query-based Detection and Segmentation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.