Research map / Architectures and efficiency / CNN design
Filter design and trained-filter analysis
Redesigns CNN filters through shaping, structured filter banks, and empirical analysis of learned filters.
128 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in CNN design, keyword cluster “cnn · filter · trained”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Leveling off Several papers from 2025–26 still report on 2 tracked benchmarks, but the best result has barely moved since 2024.
12 papers in 2023–24 and 12 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 8.8% to 11.8%.
Open questions the data points to
- Papers report CNN on CIFAR-10 (accuracy (%)) anywhere from 69.76 to 90.86: comparisons against it depend on whose number you take.
What looks settled
- CIFAR-10 (accuracy): best up to 2024 97.19, best in 2025–26 96.3 from 3 papers, at most 0.2% better, if at all.
- CIFAR-100 (accuracy): best up to 2024 78.48, best in 2025–26 77.02 from 2 papers, at most 0.2% better, if at all.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · NetVLAD: CNN Architecture for Weakly Supervised Place Recognition (CVPR 2016 · 2,686 citations)
- 2017 · Densely Connected Convolutional Networks (CVPR 2017 · 45,904 citations)
- 2018 · Interpretable Convolutional Neural Networks (CVPR 2018 · 820 citations)
- 2019 · ADCrowdNet: An Attention-Injective Deformable Convolutional Network for Crowd Understanding (CVPR 2019 · 349 citations)
- 2020 · Wavelet Integrated CNNs for Noise-Robust Image Classification (CVPR 2020 · 254 citations)
- 2021 · MLP-Mixer: An all-MLP Architecture for Vision (NeurIPS 2021 · 1,429 citations)
- 2022 · Topology-Aware Convolutional Neural Network for Efficient Skeleton-Based Action Recognition (AAAI 2022 · 177 citations)
- 2023 · MSDC: Exploiting Multi-State Power Consumption in Non-intrusive Load Monitoring Based on a Dual-CNN Model (AAAI 2023 · 23 citations)
- 2024 · Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods (CVPR 2024 · 17 citations)
- 2025 · ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression (NeurIPS 2025 · 3 citations)
- 2026 · A Theory of Contrastive Learning with Natural Images (ICML 2026 · 0 citations)
Earliest papers here
- NetVLAD: CNN Architecture for Weakly Supervised Place Recognition (CVPR 2016 · 2,686 citations)
- Recursive Recurrent Nets With Attention Modeling for OCR in the Wild (CVPR 2016 · 497 citations)
- Dynamic Filter Networks (NeurIPS 2016 · 474 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · accuracy (higher is better)
15 papers, 2017–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 93.66 | Local Binary Convolutional Neural Networks (CVPR 2017 · 301 citations) |
| 2018 | 94.05 | Perturbative Neural Networks (CVPR 2018 · 38 citations) |
| 2019 | 95.39 | Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation (AAAI 2019 · 102 citations) |
| 2020 | 97.19 | FSNet: Compression of Deep Convolutional Neural Networks by Filter Summary (ICLR 2020 · 9 citations) |
CIFAR-100 · accuracy (higher is better)
5 papers, 2019–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 78.13 | Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation (AAAI 2019 · 102 citations) |
| 2022 | 78.48 | Revisiting Sparse Convolutional Model for Visual Recognition (NeurIPS 2022 · 21 citations) |
MNIST · accuracy (higher is better)
4 papers, 2017–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 99.51 | Local Binary Convolutional Neural Networks (CVPR 2017 · 301 citations) |
| 2019 | 99.72 | DeepCaps: Going Deeper With Capsule Networks (CVPR 2019 · 245 citations) |
What has been tried
119 papers here report 1,244 measured changes (a component added, removed or swapped, compared with a baseline): helped 68%, no clear effect 8%, hurt 21%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use 3×3 kernels for all ConvNeXt blocks | ImageNet validation (first 5,000 images) · subexplanations ≥50% | 5360.43 -> 3089.83 | Hurt | Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods (CVPR 2024 · 17 citations) |
| Use Bernoulli sampling instead of Gumbel-Softmax for filter-cluster assignment | CIFAR-10 · ACC3 | 0.910 -> 0.333 | Helped | PICNN: A Pathway towards Interpretable Convolutional Neural Networks (AAAI 2024 · 6 citations) |
| Add the class token | CUB-200-2011 · accuracy | 84.32 ± 0.10 -> 85.54 ± 0.20 | Helped | Interpretable Image Classification with Adaptive Prototype-based Vision Transformers (NeurIPS 2024 · 6 citations) |
| train the CNN with Elite BackProp (EBP) | P2, P3.1, P3.2, P3.3, P5, P10, GT43, MS · rule-set size | 28 ± 5 -> 12 ± 5 (P2); 49 ± 9 -> 16 ± 5 (P3.1); 26 ± 7 -> 7 ± 1 (P3.2); 60 ± 11 -> 23 ± 4 (P3.3); 131 ± 10 -> 30 ± 8 (P5 | Helped | NeSyFOLD: A Framework for Interpretable Image Classification (AAAI 2024 · 5 citations) |
| use argmax instead of summed softmax with a 0.5 threshold | ImageNet16-like, local shape suppression · relative accuracy | 0.276 -> 0.361 | Helped | ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression (NeurIPS 2025 · 3 citations) |
| add learnable positional embeddings | Beauty, Sports, Yelp, LastFM, ML-1M, and Foursquare · NDCG@20 | 0.0705 -> 0.0702; 0.0425 -> 0.0396; 0.0321 -> 0.0320; 0.0572 -> 0.0496; 0.1955 -> 0.1936; 0.0176 -> 0.0160 | Mixed | TV-Rec: Time-Variant Convolutional Filter for Sequential Recommendation (NeurIPS 2025 · 2 citations) |
| Shuffle transferred filters across layers | ImageNet semantically divided (man-made to natural) · accuracy | 86.9% -> 86.2% | Hurt | The Master Key Filters Hypothesis: Deep Filters Are General (AAAI 2025 · 1 citation) |
| Add parameter scaling to padding alignment | CIFAR-10 + CIFAR-100, tasks 0 + 0 · accuracy | 50.00 -> 87.00; 37.00% | Helped | Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks (NeurIPS 2024 · 1 citation) |
| remove the skip connections from Pool Skip | CIFAR100 · Top-1 error (%) | 27.23 -> 44.07 | Hurt | Beyond Skip Connection: Pooling and Unpooling Design for Elimination Singularities (AAAI 2025 · 0 citations) |
| Replace standard global pooling with iterative global pooling | 7 × 7 global pooling · RAM usage | 2\% of the original size | Helped | msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML (NeurIPS 2025 · 0 citations) |
| Use impulse rather than box spatial filters | CIFAR-10 · accuracy | 81.70 -> 90.68 | Helped | Structured Initialization for Vision Transformers (NeurIPS 2025 · 0 citations) |
| Replace trained depthwise filters with linear-shift approximations from 50 candidate filters | ImageNet-1k · accuracy | 83.0% -> 75.4% (ConvNeXtv2 Tiny); 82.3% -> 79.4% (HorNet Tiny) | Hurt | The Quest for Universal Master Key Filters in DS-CNNs (NeurIPS 2025 · 0 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| CNN | CIFAR-10 | accuracy (%) | 69.76 – 90.86 | 2 papers, e.g. NeurIPS 2025 and NeurIPS 2019 |
What reviewers will expect
Datasets and baselines named in the 9 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 5 of 9 papers
- STL-10 · 4 of 9 papers
- CIFAR-100 · 3 of 9 papers
- ImageNet · 3 of 9 papers
- ImageNet-1K · 3 of 9 papers
- Food 101 · 2 of 9 papers
- Oxford-IIIT Pets · 2 of 9 papers
- Oxford 102 Flowers · 2 of 9 papers
Baselines
- ResNet18 · 2 of 9 papers
- ResNet34 · 2 of 9 papers
- ViT · 2 of 9 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 24 papers; from OpenAlex and the papers' affiliations).
- Massachusetts Institute of Technology · 3 papers
- Carnegie Mellon University · 2 papers
- TU Wien Vienna University of Technology · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Compact CNN architectures (114 papers)
- Theory of convolutional inductive bias (64 papers)
- CNN filter pruning (43 papers)
- Biologically inspired CNN explanation (46 papers)
Elsewhere in the topic
- Efficient CNN scaling and dynamic inference (104 papers)
- Dense matching and estimation CNNs (93 papers)
- Invertible and wide-network analysis (65 papers)
- Large and dynamic kernel design (62 papers)
- Group symmetry equivariant networks (55 papers)
- Steerable rotation-equivariant CNNs (40 papers)
- Higher-order pooling (20 papers)
- Group convolution theory (19 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).