Research map / Architectures and efficiency / RNNs and normalization
Model Rubik’s Cube: Twisting Resolution, Depth and Width for TinyNets
Kai Han, Yunhe Wang, Qiulin Zhang, Wei Zhang, Chunjing XU, Tong Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| shrink with tiny formula (r,d first, width from FLOPs) instead of inverse EfficientNet compound rule | EfficientNet-B0 at 200M FLOPs · ImageNet-100 | top-1 accuracy: 75.8 -> 77.6 | Improved |
| tiny formula instead of inverse compound rule | EfficientNet-B0 at 100M FLOPs · ImageNet-100 | top-1 accuracy: 72.1 -> 74.1 | Improved |
| tiny formula vs shrinking only resolution (ablation) | EfficientNet-B0 200M · ImageNet-100 | top-1 accuracy: 74.9 -> 77.6 | Improved |
| tiny formula vs shrinking only width (ablation) | EfficientNet-B0 200M · ImageNet-100 | top-1 accuracy: 77.2 -> 77.6 | Improved |
| tiny formula instead of inverse compound rule | EfficientNet-B0 at 24M FLOPs · ImageNet-1000 | top-1 accuracy: 56.7 -> 59.9 | Improved |
| tiny formula instead of inverse compound rule | EfficientNet-B0 at 339M vs 387M · ImageNet-1000 | top-1 accuracy: 76.7 -> 76.8 | No clear effect |
| add RandAugment | TinyNet-A · ImageNet-1000 | top-1 accuracy: 76.8 -> 77.7 | Improved |
| add RandAugment | EfficientNet-B0 · ImageNet-1000 | top-1 accuracy: 76.7 -> 77.7 | Improved |
| tiny formula instead of EfficientNet rule on ResNet-50 | ResNet-50 at 1.8B FLOPs · ImageNet-100 | top-1 accuracy: 76.9 -> 78.2 | Improved |
| tiny formula instead of EfficientNet rule on ResNet-50 | ResNet-50 at 3.6B FLOPs · ImageNet-100 | top-1 accuracy: 78.3 -> 79.3 | Improved |
| swap EfficientNet-B-3 backbone for TinyNet-D | SSDLite · MS COCO | mAP: 17.1 -> 19.2 | Improved |
| tiny formula vs width-only shrink | GhostNet at 300M FLOPs · ImageNet-1000 | top-1 accuracy: 76.4 -> 77.0 | Improved |
Closest papers by meaning
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (ICML 2019 · similarity 0.85)
- DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks (ICML 2022 · similarity 0.84)
- Fast and Accurate Binary Neural Networks Based on Depth-Width Reshaping (AAAI 2023 · similarity 0.84)
- The Expressive Power of Neural Networks: A View from the Width (NeurIPS 2017 · similarity 0.84)
- Wide Compression: Tensor Ring Nets (CVPR 2018 · similarity 0.84)
- Truncating Wide Networks Using Binary Tree Architectures (ICCV 2017 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
