Research map / Architectures and efficiency / CNN design
Efficient CNN scaling and dynamic inference
Scales and compresses CNNs with compound scaling, dynamic spatial computation, and channel-wise convolutions.
104 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in CNN design, keyword cluster “convolution · kernel · spatial”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
13 papers in 2023–24 and 4 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 9.5% to 3.9%.
No longer reported by papers here: ImageNet (Top-1 accuracy, last in 2022); CIFAR-10 (accuracy, last in 2020); CIFAR-100 (accuracy, last in 2022); CIFAR-10 (top-1 accuracy (%), last in 2023); ImageNet-1K val (top-1 accuracy, last in 2024); ImageNet-1k (Top-1 accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Rethinking the Inception Architecture for Computer Vision (CVPR 2016 · 31,549 citations)
- 2017 · Xception: Deep Learning With Depthwise Separable Convolutions (CVPR 2017 · 19,596 citations)
- 2018 · Squeeze-and-Excitation Networks (CVPR 2018 · 30,472 citations)
- 2019 · EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (ICML 2019 · 4,992 citations)
- 2020 · GhostNet: More Features From Cheap Operations (CVPR 2020 · 4,776 citations)
- 2021 · RepVGG: Making VGG-Style ConvNets Great Again (CVPR 2021 · 2,628 citations)
- 2022 · GhostNetV2: Enhance Cheap Operation with Long-Range Attention (NeurIPS 2022 · 278 citations)
- 2023 · ConvNeXt V2: Co-Designing and Scaling ConvNets With Masked Autoencoders (CVPR 2023 · 1,589 citations)
- 2024 · InceptionNeXt: When Inception Meets ConvNeXt (CVPR 2024 · 401 citations)
- 2025 · Are Spatial-Temporal Graph Convolution Networks for Human Action Recognition Over-Parameterized? (CVPR 2025 · 13 citations)
- 2026 · Convolutional Learnable-Group Weightless Neural Network (ICML 2026)
Earliest papers here
- Rethinking the Inception Architecture for Computer Vision (CVPR 2016 · 31,549 citations)
- Accurate Image Super-Resolution Using Very Deep Convolutional Networks (CVPR 2016 · 7,779 citations)
- Convolutional Neural Fabrics (NeurIPS 2016 · 170 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ImageNet · Top-1 accuracy (higher is better)
9 papers, 2018–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 72.2 | Constructing Fast Network through Deconstruction of Convolution (NeurIPS 2018 · 23 citations) |
| 2019 | 76.5 | All You Need Is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification (CVPR 2019 · 89 citations) |
| 2021 | 82.7 | Container: Context Aggregation Networks (NeurIPS 2021 · 18 citations) |
CIFAR-10 · accuracy (higher is better)
8 papers, 2016–2020 · not reported since 2020
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 92.18 | Deep SimNets (CVPR 2016 · 21 citations) |
| 2017 | 95.4 | Building a Regular Decision Boundary With Deep Networks (CVPR 2017 · 17 citations) |
CIFAR-100 · accuracy (higher is better)
8 papers, 2017–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 79.6 | Building a Regular Decision Boundary With Deep Networks (CVPR 2017 · 17 citations) |
| 2019 | 91.7 | EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (ICML 2019 · 4,992 citations) |
CIFAR-10 · top-1 accuracy (%) (higher is better)
5 papers, 2019–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 98 | Do Better ImageNet Models Transfer Better? (CVPR 2019 · 1,284 citations) |
ImageNet-1K val · top-1 accuracy (higher is better)
5 papers, 2021–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 79 | Dynamic Resolution Network (NeurIPS 2021 · 1 citation) |
| 2022 | 82.3 | Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets (NeurIPS 2022 · 39 citations) |
| 2023 | 88.9 | ConvNeXt V2: Co-Designing and Scaling ConvNets With Masked Autoencoders (CVPR 2023 · 1,589 citations) |
ImageNet-1k · Top-1 accuracy (higher is better)
5 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 60.9 | Binarizing MobileNet via Evolution-Based Searching (CVPR 2020 · 49 citations) |
| 2021 | 78.65 | SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks (ICML 2021 · 652 citations) |
| 2022 | 79.9 | BiMLP: Compact Binary Architectures for Vision Multi-Layer Perceptrons (NeurIPS 2022 · 5 citations) |
| 2024 | 85.24 | AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generation (NeurIPS 2024 · 1 citation) |
What has been tried
98 papers here report 1,081 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 7%, hurt 19%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add the horizontal band-kernel branch | ImageNet-1K val · top-1 accuracy | 81.9 -> 82.3 | Helped | InceptionNeXt: When Inception Meets ConvNeXt (CVPR 2024 · 401 citations) |
| add the information-compression penalty loss during warm-up | NTU120-Xset · top-1 accuracy | Helped | Are Spatial-Temporal Graph Convolution Networks for Human Action Recognition Over-Parameterized? (CVPR 2025 · 13 citations) | |
| replace clustering with convolution | Kodak · coding performance | Hurt | Another Way to the Top: Exploit Contextual Clustering in Learned Image Coding (AAAI 2024 · 11 citations) | |
| PiX with zeta=8 squeezing on ResNet-50 | ImageNet · Top-1 accuracy | +0.81% over zeta=8 baseline with ~56% fewer FLOPs | Helped | Pick-or-Mix: Dynamic Channel Sampling for ConvNets (CVPR 2024 · 2 citations) |
| Use an asymmetric rather than symmetric distribution of convolution and transformer blocks | ImageNet-1K · top-1 accuracy | 82.8% -> 83.4% | Helped | AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generation (NeurIPS 2024 · 1 citation) |
| Global Average Pooling vs. global max pooling | Atari 20 games · performance | Helped | Mind the GAP! The Challenges of Scale in Pixel-based Deep Reinforcement Learning (NeurIPS 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 7 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet-1K · 2 of 7 papers
- ADE20K · 2 of 7 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 17 papers; from OpenAlex and the papers' affiliations).
- Nanjing University · 2 papers
- Meta · 2 papers
- Alibaba Group · 2 papers
Use it
Download BibTeX of 29 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Dense matching and estimation CNNs (93 papers)
- Invertible and wide-network analysis (65 papers)
- Large and dynamic kernel design (62 papers)
- Higher-order pooling (20 papers)
Elsewhere in the topic
- Filter design and trained-filter analysis (128 papers)
- Compact CNN architectures (114 papers)
- Theory of convolutional inductive bias (64 papers)
- Group symmetry equivariant networks (55 papers)
- Biologically inspired CNN explanation (46 papers)
- CNN filter pruning (43 papers)
- Steerable rotation-equivariant CNNs (40 papers)
- Group convolution theory (19 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).