Research map / Architectures and efficiency / Neural architecture search
Robustifying and Boosting Training-Free Neural Architecture Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RoBoT, which combines existing training-free NAS metrics via a weighted linear combination whose weights are tuned by Bayesian optimization, then runs greedy search on the robust metric to boost the final architecture, with a partial-monitoring regret analysis.
Problem
Training-free NAS metrics do not perform consistently across diverse tasks, and there is an estimation gap between metric rankings and true architecture performance that limits the quality of selected architectures.
Approach
Optimize a weighted linear combination of training-free metrics with Bayesian optimization to get a robust estimation metric, then exploit it with greedy search over its top-ranked architectures, with a theoretical bound on expected rank using Precision@T and partial monitoring.
Key results
- On NAS-Bench-201, RoBoT reaches 94.36 / 73.51 / 46.34 test accuracy on C10/C100/IN-16 at 3051 GPU-seconds, matching or exceeding the best single training-free metric.
- On TransNAS-Bench-101 micro, RoBoT attains validation ranking 2 on Scene and 1 on Object, matching or beating the best single training-free metric on several tasks.
- On ImageNet in the DARTS space, RoBoT reports 24.1% top-1 error with 0.6 GPU days search cost, outperforming TE-NAS (24.5%) and HNAS (24.3%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RoBoT instead of the best single training-free metric | NAS-Bench-201 · C10, C100, and IN-16 | test accuracy (%): 94.36 vs 94.36; 73.51 vs 73.51; 46.34 vs 46.34 | No clear effect |
| Use RoBoT instead of the best single training-free metric | TransNAS-Bench-101 micro · Scene, Object, Jigsaw, Layout, Segment., Normal, and Autoenco. | validation ranking: 2 vs 2; 1 vs 1; 17 vs 22; 17 vs 18; 4 vs 3; 8 vs 14; 66 vs 36 | Mixed |
| Use RoBoT instead of the best single training-free metric | TransNAS-Bench-101 macro · Scene, Object, Jigsaw, Layout, Segment., Normal, and Autoenco. | validation ranking: 1 vs 1; 6 vs 8; 3 vs 5; 6 vs 2; 2 vs 2; 2 vs 2; 5 vs 5 | Mixed |
| Use RoBoT instead of HNAS | ImageNet DARTS search space · ImageNet | Top-1 test error (%): 24.3 -> 24.1 | Helped |
| Use RoBoT instead of HNAS | ImageNet DARTS search space · ImageNet | Top-5 test error (%): 7.4 -> 7.3 | Helped |
| Combine training-free metrics with an optimized weighted linear combination and greedy search | training-free NAS across the evaluated tasks · NAS benchmarks | architecture search performance | Mixed |
Surprising findings
- The highest-scoring architecture chosen by synflow on the Object task ranks poorly, while the top-50 architectures contain the optimum, indicating a large estimation gap.
Limitations
- Relies on the assumptions of expressiveness of the hypothesis space and predictable ranking through performance, which the authors only verify empirically.
- Theoretical guarantee depends on a uniform distribution assumption on rankings within the top-T set.
- Uses proxy objective (12-epoch CIFAR-10 validation accuracy) as the objective evaluation metric rather than full training.
Experimental setup
Datasets: NAS-Bench-201 TransNAS-Bench-101 DARTS search space CIFAR-10 ImageNet
Benchmarks: NAS-Bench-201 TransNAS-Bench-101 DARTS search space
Metrics: test accuracy validation ranking test error search cost (GPU seconds) Precision@T
Baselines: REA random search REINFORCE BOHB DARTS GDAS DrNAS NASWOT TE-NAS NASI GradSign HNAS
Builds on: NASWOT Zero-Cost NAS metrics partial monitoring information directed sampling
Closest papers by meaning
- Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search (NeurIPS 2022 · similarity 0.93)
- Per-Architecture Training-Free Metric Optimization for Neural Architecture Search (NeurIPS 2025 · similarity 0.92)
- Speedy Performance Estimation for Neural Architecture Search (NeurIPS 2021 · similarity 0.91)
- Robust NAS under adversarial training: benchmark, theory, and beyond (ICLR 2024 · similarity 0.89)
- einspace: Searching for Neural Architectures from Fundamental Operations (NeurIPS 2024 · similarity 0.89)
- Towards Neural Architecture Search through Hierarchical Generative Modeling (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.