Research map / Architectures and efficiency / CNN design
Theory of convolutional inductive bias
Analyzes invariance, stability, and Lipschitz properties of convolutional layers and pooling geometry.
64 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in CNN design, keyword cluster “cnn · filter · trained”
State History Benchmarks Tried Checklist Preprints Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
12 papers in 2023–24 and 5 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 8.8% to 4.9%.
No longer reported by papers here: CIFAR-10 (accuracy, last in 2022).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Learning to Select Pre-Trained Deep Representations With Bayesian Evidence Framework (CVPR 2016 · 12 citations)
- 2017 · Variational Dropout Sparsifies Deep Neural Networks (ICML 2017 · 331 citations)
- 2018 · Feature Quantization for Defending Against Distortion of Images (CVPR 2018 · 24 citations)
- 2019 · CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks (AAAI 2019 · 130 citations)
- 2020 · On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location (CVPR 2020 · 204 citations)
- 2021 · Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs (ICLR 2021 · 81 citations)
- 2022 · Zero Stability Well Predicts Performance of Convolutional Neural Networks (AAAI 2022 · 14 citations)
- 2023 · Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations (CVPR 2023 · 9 citations)
- 2024 · Targeted Activation Penalties Help CNNs Ignore Spurious Signals (AAAI 2024 · 1 citation)
- 2025 · LayerAct: Advanced Activation Mechanism for Robust Inference of CNNs (AAAI 2025 · 1 citation)
- 2026 · Riemannian Networks over Full-Rank Correlation Matrices (ICML 2026 · 0 citations)
Earliest papers here
- Learning to Select Pre-Trained Deep Representations With Bayesian Evidence Framework (CVPR 2016 · 12 citations)
- Variational Dropout Sparsifies Deep Neural Networks (ICML 2017 · 331 citations)
- FC4: Fully Convolutional Color Constancy With Confidence-Weighted Pooling (CVPR 2017 · 241 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
4 papers, 2020–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 95.03 | Kernel Based Progressive Distillation for Adder Neural Networks (NeurIPS 2020 · 30 citations) |
What has been tried
55 papers here report 519 measured changes (a component added, removed or swapped, compared with a baseline): helped 61%, no clear effect 10%, hurt 22%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use partial streaming for only the first three convolutional layers | PPGDalia · MAE (BPM) | 4.42 -> 3.77 | Helped | Don’t Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNs (AAAI 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 6 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MNIST · 2 of 6 papers
Recent preprints
Loading recent arXiv preprints…
Use it
Download BibTeX of 28 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Filter design and trained-filter analysis (128 papers)
- Compact CNN architectures (114 papers)
- CNN filter pruning (43 papers)
- Biologically inspired CNN explanation (46 papers)
Elsewhere in the topic
- Efficient CNN scaling and dynamic inference (104 papers)
- Dense matching and estimation CNNs (93 papers)
- Invertible and wide-network analysis (65 papers)
- Large and dynamic kernel design (62 papers)
- Group symmetry equivariant networks (55 papers)
- Steerable rotation-equivariant CNNs (40 papers)
- Higher-order pooling (20 papers)
- Group convolution theory (19 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).