Research map / Architectures and efficiency / Neural architecture search
Bi-level differentiable NAS
Reformulates gradient-based architecture search optimization with sampling, mixed-level, and geometry-aware methods.
84 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in Neural architecture search, keyword cluster “search nas · bench · architecture search”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Leveling off Several papers from 2025–26 still report on 2 tracked benchmarks, but the best result has barely moved since 2024.
23 papers in 2023–24 and 15 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 30.7% to 30.0%.
What looks settled
- CIFAR-10 (accuracy (%)): best up to 2024 97.04, best in 2025–26 97.56 from 2 papers, less than 1% better.
- ImageNet (Top-1 accuracy (%)): best up to 2024 80, best in 2025–26 76.1 from 2 papers, at most 0.2% better, if at all.
No longer reported by papers here: CIFAR-10 (test error (%), last in 2024); NAS-Bench-201 CIFAR-10 test (accuracy, last in 2023); NAS-Bench-201 CIFAR-100 test (accuracy, last in 2023); NAS-Bench-201 ImageNet-16-120 test (accuracy (%), last in 2023).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Understanding and Simplifying One-Shot Architecture Search (ICML 2018 · 528 citations)
- 2019 · Regularized Evolution for Image Classifier Architecture Search (AAAI 2019 · 2,692 citations)
- 2020 · MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation (CVPR 2020 · 131 citations)
- 2021 · BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (AAAI 2021 · 259 citations)
- 2022 · b-DARTS: Beta-Decay Regularization for Differentiable Architecture Search (CVPR 2022 · 112 citations)
- 2023 · EvoPrompting: Language Models for Code-Level Neural Architecture Search (NeurIPS 2023 · 18 citations)
- 2024 · DC-NAS: Divide-and-Conquer Neural Architecture Search for Multi-Modal Classification (AAAI 2024 · 21 citations)
- 2025 · Design Principle Transfer in Neural Architecture Search via Large Language Models (AAAI 2025 · 13 citations)
- 2026 · pTNAS: Progressive Neural Architecture Search for Tabular Data (ICML 2026 · 0 citations)
Earliest papers here
- Understanding and Simplifying One-Shot Architecture Search (ICML 2018 · 528 citations)
- Regularized Evolution for Image Classifier Architecture Search (AAAI 2019 · 2,692 citations)
- SNAS: stochastic neural architecture search (ICLR 2019 · 276 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CIFAR-10 · test error (%) (lower is better)
8 papers, 2019–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 2.51 | Efficient Forward Architecture Search (NeurIPS 2019 · 9 citations) |
| 2020 | 1.98 | Posterior-Guided Neural Architecture Search (AAAI 2020 · 7 citations) |
CIFAR-10 · accuracy (%) (higher is better)
5 papers, 2019–2025 · small gains only
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 93.67 | TAPAS: Train-Less Accuracy Predictor for Architecture Search (AAAI 2019 · 64 citations) |
| 2023 | 97.04 | EMT-NAS:Transferring Architectural Knowledge Between Tasks From Different Datasets (CVPR 2023 · 17 citations) |
| 2025 | 97.56 | Prior Knowledge Guided Neural Architecture Generation (ICML 2025) |
ImageNet · Top-1 accuracy (%) (higher is better)
5 papers, 2021–2025 · flat since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 80 | One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking (CVPR 2021 · 30 citations) |
NAS-Bench-201 CIFAR-10 test · accuracy (higher is better)
5 papers, 2021–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 93.58 | iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · 16 citations) |
| 2022 | 94.37 | Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search (CVPR 2022 · 62 citations) |
NAS-Bench-201 CIFAR-100 test · accuracy (higher is better)
5 papers, 2021–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 70.83 | iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · 16 citations) |
| 2022 | 83.76 | b-DARTS: Beta-Decay Regularization for Differentiable Architecture Search (CVPR 2022 · 112 citations) |
NAS-Bench-201 ImageNet-16-120 test · accuracy (%) (higher is better)
5 papers, 2021–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 40.89 | iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · 16 citations) |
| 2022 | 46.85 | Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search (CVPR 2022 · 62 citations) |
What has been tried
70 papers here report 791 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 8%, hurt 20%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Train all individuals on the entire training dataset instead of dividing the population and data | NTU RGB-D · accuracy (%) | 90.85±0.05 -> 90.86±0.03 | No clear effect | DC-NAS: Divide-and-Conquer Neural Architecture Search for Multi-Modal Classification (AAAI 2024 · 21 citations) |
| Add the shrinking pipeline | NAS-Bench-201 · accuracy | 93.31 -> 94.36 | Helped | IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance (AAAI 2024 · 20 citations) |
| Add the elitism-preserve strategy to evolutionary proxy search | Proxy search across datasets · search efficiency and proxy results | Helped | Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery (AAAI 2024 · 19 citations) | |
| omit principle adaptation | Trans101 tasks · model rank | Mixed | Design Principle Transfer in Neural Architecture Search via Large Language Models (AAAI 2025 · 13 citations) | |
| add fine-grained subgraph selection for each query | DrugBank S0 · reported performance | 86.31\std 0.36 -> 92.08\std 0.22 | Helped | Customized Subgraph Selection and Encoding for Drug-drug Interaction Prediction (NeurIPS 2024 · 9 citations) |
| Add supernet shifting during evolutionary search | ImageNet-1K · Top-1 accuracy | 74.01 -> 74.28 | Helped | Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach (CVPR 2024 · 5 citations) |
| Use MGD updates instead of mean, sequential, or single-task MC-sample gradient updates | NAS-Bench-201 · hypervolume | Helped | Multi-objective Differentiable Neural Architecture Search (ICLR 2025 · 2 citations) | |
| search a LaMOO-selected region rather than the entire search space | HW-NASBench · average accuracy | 68.28 -> 72.12 | Helped | CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Framework (NeurIPS 2024 · 2 citations) |
| Remove importance normalization | ImageNet · Top-1 (%) | 70.7 -> 0.1 | Hurt | Differentiable Model Scaling using Differentiable Topk (ICML 2024 · 1 citation) |
| increase number of heads to 128 | ImageNet-R (IN-100 trained) · OoD accuracy | 41.58 -> 43.47 | Helped | Vision Transformer Neural Architecture Search for Out-of-Distribution Generalization: Benchmark and Insights (NeurIPS 2024 · 1 citation) |
| use full-width and full-depth synthesis instead of motif synthesis | language modeling · downstream performance | Helped | STAR: Synthesis of Tailored Architectures (ICLR 2025 · 0 citations) | |
| Add adapters at all layers to a frozen backbone | Meta-Dataset single-domain · average accuracy | 67.8 -> 70.4 | Helped | Neural Fine-Tuning Search for Few-Shot Learning (ICLR 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 10 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- NAS-Bench-201 · 3 of 10 papers
- CIFAR-10 · 2 of 10 papers
Baselines
- REA · 3 of 10 papers
- DARTS · 3 of 10 papers
- SNAS · 2 of 10 papers
- PC-DARTS · 2 of 10 papers
- LEMONADE · 2 of 10 papers
- AlphaEvolve · 2 of 10 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 38 papers; from OpenAlex and the papers' affiliations).
- Huawei · 3 papers
- Google DeepMind · 3 papers
- Samsung · 2 papers
- City University of Hong Kong · 2 papers
- Shanghai Jiao Tong University · 2 papers
- University of Edinburgh · 2 papers
- University of Oxford · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Efficient NAS search strategies (86 papers)
- One-shot and few-shot supernets (30 papers)
Elsewhere in the topic
- Latency-aware architecture search (91 papers)
- AutoML pipeline search (57 papers)
- Differentiable architecture search (DARTS) (32 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).