Research map / Architectures and efficiency / Neural architecture search
NAS-Bench-x11 and the Power of Learning Curves
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add NAS-Bench-101 accuracy features to NAS-Bench-111 (ablation) | NAS-Bench-111 · NAS-Bench-111 test set | Avg. KT: 0.531 -> 0.611 | Helped |
| Add NAS-Bench-101 accuracy features to NAS-Bench-111 (ablation) | NAS-Bench-111 · NAS-Bench-111 test set | Final KT: 0.645 -> 0.794 | Helped |
| Add first-three-epoch accuracy features (ablation) | NAS-Bench-NLP11 · NAS-Bench-NLP11 test set | Avg. KT: 0.449 -> 0.862 | Helped |
| Add first-three-epoch accuracy features (ablation) | NAS-Bench-NLP11 · NAS-Bench-NLP11 test set | Final KT: 0.416 -> 0.820 | Helped |
| Replace SVD with VAE compression (ablation) | SVD-LGB-GKDE surrogate · NAS-Bench-111 test set | Avg. KT: 0.611 -> 0.462 | Hurt |
| Replace SVD with VAE compression (ablation) | SVD-LGB-GKDE surrogate · NAS-Bench-311 test set | Avg. KT: 0.728 -> 0.568 | Hurt |
| Use SVD-LGB-GKDE instead of the other tested model combinations (ablation) | NAS-Bench-111 and NAS-Bench-311 surrogates · NAS-Bench-111 and NAS-Bench-311 test sets | Avg. KT | Helped |
| Augment single-fidelity NAS algorithms with learning-curve extrapolation | BANANAS, local search, and regularized evolution · Four search spaces | validation accuracy | Helped |
| Use LcSVR rather than WPM for learning-curve extrapolation | BANANAS, local search, and regularized evolution · Four search spaces | validation accuracy | Helped |
| Start learning-curve extrapolation at roughly 20% of total training epochs (ablation) | SVR- and WPM-based NAS methods · Most search spaces | search performance | Helped |
Closest papers by meaning
- Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks (ICLR 2022 · similarity 0.92)
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search (ICLR 2020 · similarity 0.91)
- Generalization Guarantees for Neural Architecture Search with Train-Validation Split (ICML 2021 · similarity 0.90)
- NAS-Bench-101: Towards Reproducible Neural Architecture Search (ICML 2019 · similarity 0.90)
- Speedy Performance Estimation for Neural Architecture Search (NeurIPS 2021 · similarity 0.90)
- Robustifying and Boosting Training-Free Neural Architecture Search (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.