Research map / Architectures and efficiency / RNNs and normalization
Fast and Accurate Model Scaling
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use fast compound scaling (α=0.8) instead of uniform compound scaling | EfficientNet-B0 · ImageNet | runtime | Helped |
| Use fast compound scaling (α=0.8) instead of width-only scaling | EfficientNet-B0 · ImageNet | accuracy | Helped |
| Use α<1 rather than width-only scaling (α=1) (ablation) | EfficientNet-B0 · ImageNet | error | Helped |
| Vary α among tested values below 1 (ablation) | EfficientNet-B0 · ImageNet | error | No clear effect |
| Use group convolution rather than depthwise convolution (ablation) | RegNetZ · ImageNet | accuracy | Helped |
| Replace group convolution with depthwise convolution (ablation) | RegNetZ · ImageNet | accuracy | Hurt |
| Scale an intermediate 4GF model rather than a 500MF model to 16GF | RegNetY · ImageNet | error: 0.3% | Helped |
| Use random-search optimized models rather than scaled models at 4GF | RegNetY and RegNetZ · ImageNet | error: 0.6%; 0.1% | Helped |
| Scale width primarily, with smaller increases in depth and resolution | Convolutional networks · ImageNet | activations: O(√s) vs O(s) | Helped |
| Use activations rather than flops as a runtime proxy when scaling one model | Scaled models · ImageNet | runtime correlation: r ≥ 0.99 vs 0.81 | Helped |
Closest papers by meaning
- Principled Architecture-aware Scaling of Hyperparameters (ICLR 2024 · similarity 0.88)
- A Dynamical Model of Neural Scaling Laws (ICML 2024 · similarity 0.87)
- Scaling Exponents Across Parameterizations and Optimizers (ICML 2024 · similarity 0.87)
- 4+3 Phases of Compute-Optimal Neural Scaling Laws (NeurIPS 2024 · similarity 0.87)
- Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks (ICLR 2024 · similarity 0.87)
- Kinetics: Rethinking Test-Time Scaling Law (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.