Research map / Architectures and efficiency / RNNs and normalization
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Tien-Ju Yang, Andrew Howard, Bo Chen, Xiao Zhang, Alec Go, Mark Sandler, Vivienne Sze, Hartwig Adam
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt with NetAdapt instead of MorphNet, small MobileNetV1 on mobile CPU (latency-guided) | small MobileNetV1 · ImageNet, Pixel 1 CPU | speed and top-1 accuracy: 1.6x faster with 0.3% higher accuracy | Improved |
| adapt with NetAdapt instead of width/resolution multipliers, small MobileNetV1 | small MobileNetV1 · ImageNet, Pixel 1 CPU | speed at same accuracy: up to 1.7x faster with the same or higher accuracy | Improved |
| adapt with NetAdapt instead of multipliers and ADC, large MobileNetV1 on mobile CPU | large MobileNetV1 · ImageNet, Pixel 1 CPU | speed: 1.4x and 1.2x speed-up | Improved |
| adapt with NetAdapt on mobile GPU instead of benchmark algorithms, large MobileNetV1 | large MobileNetV1 · ImageNet, Galaxy S8 GPU | speed: up to 1.2x speed-up with higher accuracy | Improved |
| add dropout and label smoothing to the training configuration of NetAdapt networks | NetAdapt-adapted large MobileNetV1 · ImageNet, Pixel 1 CPU | top-1 accuracy: increased by 1.3% | Improved |
| guide NetAdapt by latency (direct metric) rather than MACs (ablation) | NetAdapt · ImageNet, Pixel 1 CPU, MACs-targeted (Table 1) | latency: MorphNet 6.52 ms (140%) vs NetAdapt 6.01 ms (129%); fewer MACs (81%) did not give lower latency than baseline 4.65 ms | Improved |
| add short-term fine-tuning (ten thousand iterations) to each network proposal (ablation) | NetAdapt on small MobileNetV1 · ImageNet, no long-term fine-tune | top-1 accuracy: accuracy drops to nearly zero without it; above 20% with ten thousand iterations | Improved |
| increase short-term fine-tuning from ten thousand to more iterations (ablation) | NetAdapt on small MobileNetV1 · ImageNet | top-1 accuracy | Improved |
| add long-term fine-tuning at the end (ablation) | NetAdapt on small MobileNetV1 · ImageNet, Pixel 1 CPU | top-1 accuracy: up to 4.5% or 3.4% on average (range 1.8% to 4.5%) | Improved |
| use resource reduction decay 0.96 instead of 1.0 (initial 0.5 ms) (ablation) | NetAdapt on small MobileNetV1 · ImageNet, Table 2 | top-1 accuracy: 47.4 -> 47.7 (28 vs 20 iterations) | Improved |
| use larger initial resource reduction 0.8 ms (decay 0.95) instead of 0.5 ms (decay 0.96) (ablation) | NetAdapt on small MobileNetV1 · ImageNet, Table 2 | top-1 accuracy: 47.7 -> 46.7 | Hurt |
| adapt MobileNetV2 with NetAdapt instead of multipliers (similar latency) | 75% MobileNetV2 (224) · ImageNet, Pixel 1 CPU | top-1 accuracy: 69.8 -> 70.9 | Improved |
Later papers that compared with this method
5 later accepted papers report NetAdapt as a baseline in their results tables; 4 of them report a better number on most of the results they share with it.
- DepGraph: Towards Any Structural Pruning (CVPR 2023 · better on 0 of 1 shared result)
- NetAdaptV2: Efficient Neural Architecture Search With Fast Super-Network Training and Architecture Optimization (CVPR 2021 · better on 1 of 1 shared result)
- Energy-Constrained Compression for Deep Neural Networks via Weighted Sparse Projection and Layer Input Masking (ICLR 2019 · better on 2 of 2 shared results)
- MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning (ICCV 2019 · better on 1 of 1 shared result)
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices (ECCV 2018 · better on 2 of 2 shared results)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- MorphNet — MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks (CVPR 2018) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MnasNet: Platform-Aware Neural Architecture Search for Mobile (CVPR 2019 · similarity 0.86)
- ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation (CVPR 2019 · similarity 0.85)
- ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices (ICCV 2023 · similarity 0.84)
- Neural Architecture Transfer (TPAMI 2021 · similarity 0.84)
- Adapting Neural Architectures Between Domains (NeurIPS 2020 · similarity 0.84)
- NetAdaptV2: Efficient Neural Architecture Search With Fast Super-Network Training and Architecture Optimization (CVPR 2021 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
