Research map / Detection and segmentation / CNN features
Efficient network training and design
Make CNNs compact and resource-efficient via slimming, over-parameterization, and channel attention.
37 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2025) · an approach in CNN features, keyword cluster “vision · imagenet · transformers”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
7 papers in 2023–24 and 4 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 2.1% to 1.2%.
No longer reported by papers here: ImageNet val (top-1 accuracy (%), last in 2023).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Searching for Efficient Multi-Scale Architectures for Dense Image Prediction (NeurIPS 2018 · 348 citations)
- 2019 · Slimmable Neural Networks (ICLR 2019 · 231 citations)
- 2020 · ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (CVPR 2020 · 8,780 citations)
- 2021 · Coordinate Attention for Efficient Mobile Network Design (CVPR 2021 · 6,279 citations)
- 2022 · Smooth Maximum Unit: Smooth Activation Function for Deep Networks Using Smoothing Maximum Technique (CVPR 2022 · 66 citations)
- 2023 · MobileOne: An Improved One Millisecond Mobile Backbone (CVPR 2023 · 376 citations)
- 2024 · MogaNet: Multi-order Gated Aggregation Network (ICLR 2024 · 24 citations)
- 2025 · Dust-Mamba: An Efficient Dust Storm Detection Network with Multiple Data Sources (AAAI 2025 · 0 citations)
Earliest papers here
- Searching for Efficient Multi-Scale Architectures for Dense Image Prediction (NeurIPS 2018 · 348 citations)
- Learning Strict Identity Mappings in Deep Residual Networks (CVPR 2018 · 69 citations)
- Slimmable Neural Networks (ICLR 2019 · 231 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ImageNet val · top-1 accuracy (%) (higher is better)
6 papers, 2019–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 73.13 | Composite Binary Decomposition Networks (AAAI 2019 · 2 citations) |
| 2020 | 79.67 | GradAug: A New Regularization Method for Deep Neural Networks (NeurIPS 2020 · 26 citations) |
| 2023 | 89.2 | Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information (CVPR 2023 · 48 citations) |
What has been tried
36 papers here report 453 measured changes (a component added, removed or swapped, compared with a baseline): helped 75%, no clear effect 5%, hurt 16%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add the channel aggregation module CA with r = 4 | ImageNet-1K · performance | 0.6% | Helped | MogaNet: Multi-order Gated Aggregation Network (ICLR 2024 · 24 citations) |
| Replace M1 alone with M2 alone | COCO val2017, Faster R-CNN, ResNet-50 · AP | 36.2 -> 37.6 | Helped | MCA: Moment Channel Attention Networks (AAAI 2024 · 15 citations) |
| Add MRDF to Unet++ | LSDSSIMR test · Dice | 0.9132 -> 0.9286 | Helped | Dust-Mamba: An Efficient Dust Storm Detection Network with Multiple Data Sources (AAAI 2025 · 0 citations) |
| Distill outputs from all three stages instead of only the last stage | VGG16 · performance | 74.67 -> 76.69 | Helped | Inheriting Generalized Learngene for Efficient Knowledge Transfer across Multiple Tasks (AAAI 2025 · 0 citations) |
| use LR=0.01 instead of LR=0.02 | MS-COCO · test mIoU | 64.98±0.129 -> 65.98±0.124 | Helped | Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics (NeurIPS 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 6 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet-1K · 3 of 6 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 11 papers; from OpenAlex and the papers' affiliations).
- Nanjing University · 2 papers
- Zhejiang University · 2 papers
- Megvii · 2 papers
Use it
Download BibTeX of 25 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Convolution-transformer backbones (101 papers)
- Plain and high-resolution ViTs (69 papers)
- Vision state space models (13 papers)
Elsewhere in the topic
- Context modeling for semantic segmentation (241 papers)
- Region-based and query-based detectors (183 papers)
- Local features and context in CNNs (170 papers)
- Context for hard detection conditions (133 papers)
- Domain-specific segmentation and detection (118 papers)
- Box regression and loss design (94 papers)
- Efficient DETR query design (85 papers)
- Boundary-aware instance segmentation (83 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).