Research map / Architectures and efficiency / CNN design
Compact CNN architectures
Condenses and refines convolutional networks via binary structure, learned shapes, and PDE views.
114 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in CNN design, keyword cluster “cnn · filter · trained”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
18 papers in 2023–24 and 2 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 13.1% to 2.0%.
No longer reported by papers here: CIFAR-10 (accuracy, last in 2024); CIFAR-100 (accuracy, last in 2024); ImageNet (top-1 accuracy (%), last in 2021); ImageNet validation (top-1 accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Quantized Convolutional Neural Networks for Mobile Devices (CVPR 2016 · 1,258 citations)
- 2017 · Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (AAAI 2017 · 12,687 citations)
- 2018 · DropBlock: A regularization method for convolutional networks (NeurIPS 2018 · 509 citations)
- 2019 · Meta-Learning Convolutional Neural Architectures for Multi-Target Concrete Defect Classification With the COncrete DEfect BRidge IMage Dataset (CVPR 2019 · 170 citations)
- 2020 · Attention Scaling for Crowd Counting (CVPR 2020 · 331 citations)
- 2021 · An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021 · 21,812 citations)
- 2022 · CMT: Convolutional Neural Networks Meet Vision Transformers (CVPR 2022 · 903 citations)
- 2023 · DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network (CVPR 2023 · 43 citations)
- 2024 · RepViT: Revisiting Mobile CNN From ViT Perspective (CVPR 2024 · 30 citations)
- 2025 · Normalize Filters! Classical Wisdom for Deep Vision (NeurIPS 2025 · 0 citations)
- 2026 · AirWino: Optimized Winograd Convolution for Accelerating CNN Inference on ARMv8 Processors (AAAI 2026 · 0 citations)
Earliest papers here
- Quantized Convolutional Neural Networks for Mobile Devices (CVPR 2016 · 1,258 citations)
- Fast Algorithms for Convolutional Neural Networks (CVPR 2016 · 943 citations)
- ASP Vision: Optically Computing the First Layer of Convolutional Neural Networks Using Angle Sensitive Pixels (CVPR 2016 · 59 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
13 papers, 2017–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 95.01 | Deep Hyperspherical Learning (NeurIPS 2017 · 79 citations) |
| 2021 | 99.5 | An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021 · 21,812 citations) |
CIFAR-100 · accuracy (higher is better)
12 papers, 2017–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 76.39 | Deep Hyperspherical Learning (NeurIPS 2017 · 79 citations) |
| 2021 | 94.55 | An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021 · 21,812 citations) |
ImageNet · top-1 accuracy (%) (higher is better)
5 papers, 2018–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 64.3 | From Hashing to CNNs: Training Binary Weight Networks via Hashing (AAAI 2018 · 78 citations) |
| 2019 | 71.5 | In-Place Zero-Space Memory Protection for CNN (NeurIPS 2019 · 0 citations) |
| 2020 | 74.9 | AdderNet: Do We Really Need Multiplications in Deep Learning? (CVPR 2020 · 195 citations) |
ImageNet validation · top-1 accuracy (higher is better)
4 papers, 2018–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 82.52 | DropBlock: A regularization method for convolutional networks (NeurIPS 2018 · 509 citations) |
What has been tried
105 papers here report 1,125 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 10%, hurt 22%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add structural re-parameterization multi-branch topology during training | ImageNet-1K · Top-1 accuracy | 78.47% -> 78.74% | Helped | RepViT: Revisiting Mobile CNN From ViT Perspective (CVPR 2024 · 30 citations) |
| Add the supplement gate to LFMUs | ImageNet100 · Top-1 accuracy | 78.40% -> 78.94%; 78.02% -> 78.66%; 79.88% -> 80.58%; 79.36% -> 81.22%; 78.34% -> 79.36%; 78.52% -> 79.98%; 78.96% -> 80 | Helped | Multiscale Low-Frequency Memory Network for Improved Feature Extraction in Convolutional Neural Networks (AAAI 2024 · 6 citations) |
| set B to 2 rather than 1 during B-cosification | ImageNet validation · GridPG localisation | 33.9{ ±0.2} -> 87.6{ ±0.2} | Helped | B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable (NeurIPS 2024 · 2 citations) |
| Add gain-bias augmentation (maximum variation ±10%) | CIFAR-10 D_S · accuracy | 38.1 -> 76.9 | Helped | Normalize Filters! Classical Wisdom for Deep Vision (NeurIPS 2025 · 0 citations) |
| Replace the SPD MLR layer with MLR in structure spaces | HDM05 · accuracy | 79.78 ± 1.42 -> 78.52 ± 1.34 | Hurt | Matrix Manifold Neural Networks++ (ICLR 2024) |
What reviewers will expect
Datasets and baselines named in the 8 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet-1K · 3 of 8 papers
- CIFAR-10 · 2 of 8 papers
- MNIST · 2 of 8 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 20 papers; from OpenAlex and the papers' affiliations).
- Amazon · 2 papers
- Swiss Federal Institute of Technology · 2 papers
Use it
Download BibTeX of 29 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Filter design and trained-filter analysis (128 papers)
- Theory of convolutional inductive bias (64 papers)
- CNN filter pruning (43 papers)
- Biologically inspired CNN explanation (46 papers)
Elsewhere in the topic
- Efficient CNN scaling and dynamic inference (104 papers)
- Dense matching and estimation CNNs (93 papers)
- Invertible and wide-network analysis (65 papers)
- Large and dynamic kernel design (62 papers)
- Group symmetry equivariant networks (55 papers)
- Steerable rotation-equivariant CNNs (40 papers)
- Higher-order pooling (20 papers)
- Group convolution theory (19 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).