Research map / Architectures and efficiency / Neural architecture search
EfficientNetV2: Smaller Models and Faster Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EfficientNetV2 architecture with training-aware NAS and scaling (ablation) | ImageNet ILSVRC2012 models · ImageNet ILSVRC2012 val | training time: 54 -> 13 | Helped |
| Pretrain on ImageNet21k before finetuning | EfficientNetV2-L · ImageNet ILSVRC2012 val | top-1 accuracy: 85.7% -> 86.8% | Helped |
| Replace MBConv with Fused-MBConv in early stages 1-3 (ablation) | EfficientNet-B4 · ImageNet | TPU throughput: 262 -> 362 imgs/sec/core | Helped |
| Replace MBConv with Fused-MBConv throughout stages 1-7 (ablation) | EfficientNet-B4 · ImageNet | top-1 accuracy: 82.8% -> 81.7% | Hurt |
| Replace MBConv with Fused-MBConv throughout stages 1-7 (ablation) | EfficientNet-B4 · ImageNet | TPU throughput: 262 -> 254 imgs/sec/core | Hurt |
| Adapt regularization strength as image size increases (ablation) | progressive resizing · ImageNet | top-1 accuracy: +0.8 | Helped |
| Adapt regularization strength to image size (ablation) | random resizing · ImageNet | top-1 accuracy: +0.7 | Helped |
| Use progressive learning rather than baseline training (ablation) | ResNet50 (224) · ImageNet | training time: 4.9h -> 3.5h | Helped |
| Use progressive learning rather than baseline training (ablation) | EfficientNet-B5 · ImageNet | top-1 accuracy: 83.7 -> 84.0 | Helped |
| Use progressive learning rather than baseline training (ablation) | EfficientNet-B5 · ImageNet | training time: 42.9h -> 15.2h | Helped |
| Increase RandAugment magnitude from 5 to 15 at image size 128 (ablation) | searched ImageNet model · ImageNet | top-1 accuracy: 78.3 -> 77.7 | Hurt |
| Increase RandAugment magnitude from 5 to 15 at image size 300 (ablation) | searched ImageNet model · ImageNet | top-1 accuracy: 82.5 -> 83.2 | Helped |
Closest papers by meaning
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (ICML 2019 · similarity 0.91)
- ESPNetv2: A Light-Weight, Power Efficient, and General Purpose Convolutional Neural Network (CVPR 2019 · similarity 0.88)
- The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes (NeurIPS 2022 · similarity 0.87)
- RePr: Improved Training of Convolutional Filters (CVPR 2019 · similarity 0.87)
- Revisiting ResNets: Improved Training and Scaling Strategies (NeurIPS 2021 · similarity 0.86)
- DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.