Research map / Architectures and efficiency / Neural architecture search
Efficient NAS search strategies
Improves architecture search with credit assignment, distillation, and generative modeling of architectures.
86 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Neural architecture search, keyword cluster “search nas · bench · architecture search”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
9 papers in 2023–24 and 8 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 12.0% to 16.0%.
Open questions the data points to
- Papers report DARTS (2nd) on CIFAR-100 test (test error (%)) anywhere from 17.54 to 20.58: comparisons against it depend on whose number you take.
No longer reported by papers here: CIFAR-10 (accuracy (%), last in 2024); CIFAR-10 test (test error (%), last in 2022); CIFAR-10 (search cost (GPU days), last in 2021); CIFAR-10 (test error (%), last in 2021); CIFAR-100 (accuracy (%), last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · NAS-Bench-101: Towards Reproducible Neural Architecture Search (ICML 2019 · 243 citations)
- 2020 · CARS: Continuous Evolution for Efficient Neural Architecture Search (CVPR 2020 · 279 citations)
- 2021 · FBNetV3: Joint Architecture-Recipe Search Using Predictor Pretraining (CVPR 2021 · 104 citations)
- 2022 · Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks (ICLR 2022 · 20 citations)
- 2023 · PINAT: A Permutation INvariance Augmented Transformer for NAS Predictor (AAAI 2023 · 19 citations)
- 2024 · Encodings for Prediction-based Neural Architecture Search (ICML 2024 · 1 citation)
- 2025 · Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · 3 citations)
- 2026 · Constraint-guided Hardware-aware NAS through Gradient Modification (ICLR 2026 · 0 citations)
Earliest papers here
- NAS-Bench-101: Towards Reproducible Neural Architecture Search (ICML 2019 · 243 citations)
- Graph HyperNetworks for Neural Architecture Search (ICLR 2019 · 114 citations)
- Meta Architecture Search (NeurIPS 2019 · 25 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
ImageNet · top-1 accuracy (higher is better)
7 papers, 2020–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 78.1 | Improving One-Shot NAS by Suppressing the Posterior Fading (CVPR 2020 · 81 citations) |
| 2021 | 82.8 | FBNetV3: Joint Architecture-Recipe Search Using Predictor Pretraining (CVPR 2021 · 104 citations) |
CIFAR-10 · accuracy (%) (higher is better)
6 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 98.3 | Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation (CVPR 2020 · 190 citations) |
CIFAR-10 test · test error (%) (lower is better)
6 papers, 2020–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 2.44 | Efficient Neural Architecture Search via Proximal Iterations (AAAI 2020 · 100 citations) |
| 2022 | 2.3 | Saliency-Aware Neural Architecture Search (NeurIPS 2022 · 2 citations) |
CIFAR-10 · search cost (GPU days) (lower is better)
4 papers, 2019–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 0.42 | Graph HyperNetworks for Neural Architecture Search (ICLR 2019 · 114 citations) |
| 2020 | 0.3 | Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search (ICML 2020 · 33 citations) |
CIFAR-10 · test error (%) (lower is better)
4 papers, 2020–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 2.16 | Neural Architecture Search Using Deep Neural Networks and Monte Carlo Tree Search (AAAI 2020 · 53 citations) |
CIFAR-100 · accuracy (%) (higher is better)
4 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 88.3 | Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation (CVPR 2020 · 190 citations) |
What has been tried
77 papers here report 778 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 9%, hurt 20%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Split by number of nonlinear functions rather than FLOPs | CIFAR10 test · Kendall's tau | 0.711 -> 0.735 | Helped | Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · 3 citations) |
| Aggregate VKDNW with Jacov, expressivity, trainability, and FLOPs | ImageNet16-120 · nDCG | 0.608 -> 0.664 | Helped | Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weights (CVPR 2025 · 2 citations) |
| Ensemble DGF and GAT in both graph-flow directions | NASBench-101, 128 training samples · Kendall-τ | 0.7088 (0.0003) -> 0.7322 (0.0002) | Helped | Encodings for Prediction-based Neural Architecture Search (ICML 2024 · 1 citation) |
| Increase the third hidden layer from 60 to 100 neurons | lMLP dataset B · test loss | 0.015 ± 0.004 -> 0.015 ± 0.003 | No clear effect | NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance (ICLR 2025 · 0 citations) |
| Add hierarchical search with CCNF to Evo(T-CET) on GraphNet | CIFAR-10 · accuracy (%) | 96.4 -> 97.4 | Helped | Towards Neural Architecture Search through Hierarchical Generative Modeling (ICML 2024) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| DARTS (2nd) | CIFAR-100 test | test error (%) | 17.54 – 20.58 | 2 papers, e.g. ICLR 2022 and NeurIPS 2022 |
What reviewers will expect
Datasets and baselines named in the 5 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 3 of 5 papers
- CIFAR-100 · 3 of 5 papers
- ImageNet16-120 · 3 of 5 papers
Baselines
- FLOPs · 2 of 5 papers
- GradNorm · 2 of 5 papers
- GraSP · 2 of 5 papers
- SNIP · 2 of 5 papers
- SynFlow · 2 of 5 papers
- ZiCo · 2 of 5 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 17 papers; from OpenAlex and the papers' affiliations).
- Cornell University · 2 papers
Use it
Download BibTeX of 33 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Bi-level differentiable NAS (84 papers)
- One-shot and few-shot supernets (30 papers)
Elsewhere in the topic
- Latency-aware architecture search (91 papers)
- AutoML pipeline search (57 papers)
- Differentiable architecture search (DARTS) (32 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).