Research map / Architectures and efficiency / RNNs and normalization
Revisiting ResNets: Improved Training and Scaling Strategies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| ResNet-RS vs EfficientNet at matched accuracy | image classification · ImageNet | TPU-v3 training latency: 3.0 -> 1.1 | Helped |
| cosine LR decay (ablation) | ResNet-200 · ImageNet | top-1: +0.3 | Helped |
| increase epochs to 350 without regularization (ablation) | ResNet-200 · ImageNet | top-1: -0.5 | Hurt |
| label smoothing (ablation) | ResNet-200 · ImageNet | top-1: +1.3 | Helped |
| RandAugment (ablation) | ResNet-200 · ImageNet | top-1: +0.4 | Helped |
| dropout on FC before decreasing weight decay (ablation) | ResNet-200 · ImageNet | top-1: -0.3 | Hurt |
| decrease weight decay when combining regularizers (ablation) | ResNet-200 · ImageNet | top-1: +1.5 | Helped |
| decrease weight decay with no regularization (ablation) | ResNet-50 · ImageNet minival | top-1: 79.7 -> 78.7 | Hurt |
| Squeeze-and-Excitation (ablation) | ResNet-200 · ImageNet | top-1: +0.7 | Helped |
| ResNet-D stem (ablation) | ResNet-200 · ImageNet | top-1: +0.5 | Helped |
| depth scaling instead of width scaling, 350-epoch regime | ResNet · ImageNet minival | top-1 | Helped |
| depth scaling instead of width scaling, 10-epoch regime | ResNet · ImageNet minival | top-1 | Hurt |
Closest papers by meaning
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (ICML 2019 · similarity 0.90)
- Multi-level Residual Networks from Dynamical Systems View (ICLR 2018 · similarity 0.90)
- ResNEsts and DenseNEsts: Block-based DNN Models with Improved Representation Guarantees (NeurIPS 2021 · similarity 0.89)
- ScaleNet: A Shallow Architecture for Scale Estimation (CVPR 2022 · similarity 0.88)
- Big Self-Supervised Models are Strong Semi-Supervised Learners (NeurIPS 2020 · similarity 0.88)
- ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.