AI Paper Atlas

Research map / Architectures and efficiency / Neural architecture search

MobileNetV4: Universal Models for the Mobile Ecosystem

Danfeng Qin, Chas H Leichner, Manolis Delakis, Marco Fornoni, Shixin Luo, Fan Yang, Weijun Wang, Colby Banbury, Chengxi Ye, Berkin Akin, Vaibhav Aggarwal, Tenghui Zhu and 2 more

ECCV 2024 · Architectures and efficiency › Neural architecture search · approach: Bi-level differentiable NAS · 851 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace Multi-Head Self-Attention with Multi-Query Attention in the last stage (ablation)MNv4-Conv-L · ImageNet-1K valTop-1 (%): 85.27 -> 85.24No clear effect
replace MHSA with MQA in the last stage (ablation)MNv4-Conv-L · Pixel 7 EdgeTPUattention latency (ms): 9.69 ms -> 5.16 msImproved
add stride-2 spatial down-sampling on keys and values in Mobile MQA (ablation)MNv4-Hybrid-M · Samsung S23 GPUlatency (ms): 7.4 -> 5.9Improved
search with full UIB instead of Inverted Bottleneck only (ablation)NAS · ImageNet-1K valTop-1 (%): 82.3 -> 83.3Improved
search with full UIB instead of ConvNext-Like only (ablation)NAS · ImageNet-1K valTop-1 (%): 83.2 -> 83.3No clear effect
use two-stage (coarse then fine) TuNAS search instead of one-stage (ablation)TuNAS · ImageNet-1K valTop-1 val (%): 81.26 -> 81.48Improved
train NAS SuperNet on JFT distillation data instead of ImageNet-1k (ablation)TuNAS · ImageNet-1K valMACs (G): 7.2G -> 6.2GImproved
mix D1 and D2 distillation datasets instead of D2 alone (ablation)MNv4-Conv-L distillation · ImageNet-1K valTop-1 (%): 84.1 -> 84.4Improved
add class-balanced JFT data D3 to the ImageNet mixture (ablation)MNv4-Conv-L distillation · ImageNet-1K valTop-1 (%): 84.4 -> 84.9Improved
train distillation student for 2000 epochs instead of 400 (ablation)MNv4-Conv-L distillation · ImageNet-1K valTop-1 (%): 84.9 -> 85.9Improved
use the enhanced distillation recipe instead of the previous SOTA distillationMNv4-Hybrid-L · ImageNet-1K valTop-1 (%): 85.7 -> 86.6Improved
use MNv4-Hybrid-M instead of MNv4-Conv-M as RetinaNet backboneMNv4 · COCO valAP: 32.6 -> 34.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy