Research map / Architectures and efficiency / CNN design
Large and dynamic kernel design
Rethinks convolution kernel size, shape, and dynamic composition to enlarge receptive fields.
62 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in CNN design, keyword cluster “convolution · kernel · spatial”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
12 papers in 2023–24 and 9 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 8.8% to 8.8%.
No longer reported by papers here: ImageNet (top-1 accuracy, last in 2024); CIFAR-100 (accuracy, last in 2022).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Fast ConvNets Using Group-Wise Brain Damage (CVPR 2016 · 399 citations)
- 2017 · Learning Adaptive Receptive Fields for Deep Image Parsing Network (CVPR 2017 · 69 citations)
- 2018 · Interleaved Structured Sparse Convolutional Neural Networks (CVPR 2018 · 140 citations)
- 2019 · Selective Kernel Networks (CVPR 2019 · 3,142 citations)
- 2020 · Dynamic Convolution: Attention Over Convolution Kernels (CVPR 2020 · 1,470 citations)
- 2021 · Dynamic Region-Aware Convolution (CVPR 2021 · 159 citations)
- 2022 · Omni-Dimensional Dynamic Convolution (ICLR 2022 · 181 citations)
- 2023 · Dilated convolution with learnable spacings (ICLR 2023 · 18 citations)
- 2024 · PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution (CVPR 2024 · 83 citations)
- 2025 · Frequency Dynamic Convolution for Dense Image Prediction (CVPR 2025 · 92 citations)
- 2026 · WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution (ICML 2026 · 0 citations)
Earliest papers here
- Fast ConvNets Using Group-Wise Brain Damage (CVPR 2016 · 399 citations)
- End-to-End Kernel Learning with Supervised Convolutional Kernel Networks (NeurIPS 2016 · 7 citations)
- Understanding the Effective Receptive Field in Deep Convolutional Neural Networks (NeurIPS 2016)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
11 papers, 2019–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 95.14 | Super Sparse Convolutional Neural Networks (AAAI 2019 · 52 citations) |
| 2020 | 96 | Neural Kernels Without Tangents (ICML 2020 · 27 citations) |
ImageNet · top-1 accuracy (higher is better)
7 papers, 2019–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 71.2 | HetConv: Heterogeneous Kernel-Based Convolutions for Deep CNNs (CVPR 2019 · 18 citations) |
| 2020 | 75.73 | Network Deconvolution (ICLR 2020 · 15 citations) |
| 2022 | 81.7 | RepMLPNet: Hierarchical Vision MLP With Re-Parameterized Locality (CVPR 2022 · 78 citations) |
| 2023 | 82.17 | Are Large Kernels Better Teachers than Transformers for ConvNets? (ICML 2023 · 1 citation) |
| 2024 | 83.9 | Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling (NeurIPS 2024 · 0 citations) |
CIFAR-100 · accuracy (higher is better)
5 papers, 2018–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 77.02 | Interleaved Structured Sparse Convolutional Neural Networks (CVPR 2018 · 140 citations) |
| 2019 | 77.67 | Super Sparse Convolutional Neural Networks (AAAI 2019 · 52 citations) |
| 2022 | 84.62 | Towards an Effective Orthogonal Dictionary Convolution Strategy (AAAI 2022 · 3 citations) |
What has been tried
57 papers here report 677 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 6%, hurt 19%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Replace stripe convolution with dense grid convolution at 51×51 | ADE20K · mIoU (%) | 46.1 -> 46.6 (+0.5) | Helped | PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution (CVPR 2024 · 83 citations) |
| Replace SLaK's large strip convolutions with laid-out standard convolutions | ImageNet-1K · accuracy | 82.5 -> 82.5 | No clear effect | ShiftwiseConv: Small Convolutional Kernel with Large Kernel Effect (CVPR 2025 · 22 citations) |
| add kernel partition | ImageNet val · Top-1 Acc (%) | 70.49 -> 74.77 | Helped | KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · 3 citations) |
| increase ConvNeXt-Tiny kernel size from 3×3 to 13×13 | ImageNet-A · top-1 accuracy | 5.20 -> 9.89 | Helped | Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness (ICML 2024 · 2 citations) |
| Use polynomial degree 4 instead of degree 2 | DVS128 at 200 ms latency · accuracy | 55.4 (0.03) -> 73.2 (0.03) | Helped | PLEIADES: Building Temporal Kernels with Orthogonal Polynomials (NeurIPS 2025 · 1 citation) |
| Use Cholesky instead of Björck and Bowie orthogonalization | Network from the cited training setup · accuracy | Hurt | An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures (ICML 2025 · 0 citations) | |
| Set regularisation strength ν to ∞, removing data-adaptive Gram matrix learning | CIFAR-10 · accuracy | 87.86 ± 0.09 -> 53.74 ± 0.38 | Hurt | Convolutional Deep Kernel Machines (ICLR 2024 · 0 citations) |
| Add multiresolution to a dense kernel | LRA ListOps · accuracy | 49.25 -> 53.15 | Helped | Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling (NeurIPS 2024 · 0 citations) |
| add stochastic kernel regularisation | CIFAR-10 test · accuracy | 93.71 ± 0.0150 -> 94.52 ± 0.0693 | Helped | Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines (NeurIPS 2024 · 0 citations) |
| Use six shifted Hilbert paths instead of the default three | ImageNet-64 · top-1 accuracy | 68.46% -> 62.23% | Hurt | One-dimensional Path Convolution (ICML 2025) |
| Use receptive field experts {3, 5, 7, 9, 11} instead of {3, 5, 7} | ImageNet-1K · Top-1 accuracy | 80.26/80.41 -> 80.43/80.62 | Helped | Receptive Fields As Experts in Convolutional Neural Architectures (ICML 2024) |
| Reduce the gated double layer from 128 to 64 channels | 8-frame-combining KPN · Test PSNR | 32.8 -> 31.9 | Hurt | Is Kernel Prediction More Powerful than Gating in Convolutional Neural Networks? (ICML 2024) |
What reviewers will expect
Datasets and baselines named in the 7 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- COCO · 2 of 7 papers
- ADE20K · 2 of 7 papers
- ImageNet-1K · 2 of 7 papers
- CIFAR-10 · 2 of 7 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 21 papers; from OpenAlex and the papers' affiliations).
- Chinese Academy of Sciences · 3 papers
- Meituan · 2 papers
- University of Bristol · 2 papers
Use it
Download BibTeX of 35 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Efficient CNN scaling and dynamic inference (104 papers)
- Dense matching and estimation CNNs (93 papers)
- Invertible and wide-network analysis (65 papers)
- Higher-order pooling (20 papers)
Elsewhere in the topic
- Filter design and trained-filter analysis (128 papers)
- Compact CNN architectures (114 papers)
- Theory of convolutional inductive bias (64 papers)
- Group symmetry equivariant networks (55 papers)
- Biologically inspired CNN explanation (46 papers)
- CNN filter pruning (43 papers)
- Steerable rotation-equivariant CNNs (40 papers)
- Group convolution theory (19 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).