Research map / Architectures and efficiency / Neural architecture search
JAHS-Bench-201: A Foundation For Research On Joint Architecture And Hyperparameter Search
Archit Bansal, Danny Stoll, Maciej Janowski, Arber Zela, Frank Hutter
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Search the joint architecture+hyperparameter space instead of HPO-only (ablation) | random search · CIFAR-10, Colorectal-Histology, Fashion-MNIST | time to equal accuracy: 33 times faster | Improved |
| Search the joint space instead of NAS-only (ablation) | random search · CIFAR-10, Colorectal-Histology, Fashion-MNIST | time to equal accuracy: 29 times faster | Improved |
| Search the joint space instead of HPO-only or NAS-only (ablation) | Bayesian Optimization · all three datasets | validation error | Improved |
| Choose width multiplier as Hyperband fidelity (ablation) | Hyperband · Colorectal-Histology | validation error | Hurt |
| Choose depth multiplier as fidelity (ablation) | model-based Hyperband (SMAC) · Colorectal-Histology | validation error | Improved |
| Use epochs, depth, resolution or width as the single fidelity (ablation) | Hyperband · CIFAR-10 and Fashion-MNIST | validation error | No clear effect |
| Traverse the fidelity hypercube diagonally (multi multi-fidelity) instead of single fidelity (ablation) | Hyperband · all three datasets | validation error | No clear effect |
| Diagonal fidelity traversal with model-based search (ablation) | SMAC Hyperband · all three datasets | validation error | Hurt |
| Replace training-based runs with surrogate-based runs | random search · CIFAR-10 | compute cost: 5.0 GPU-d -> 2.8 CPU-s | Improved |
| Surrogate quality on noisy Colorectal-Histology data | XGBoost surrogate · Colorectal-Histology Test-Accuracy | Kendall's tau: 0.892 -> 0.691 | Hurt |
Closest papers by meaning
- NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks (NeurIPS 2022 · similarity 0.89)
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search (ICLR 2020 · similarity 0.89)
- Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarks (ICLR 2022 · similarity 0.89)
- NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size (TPAMI 2021 · similarity 0.89)
- NAS-Bench-101: Towards Reproducible Neural Architecture Search (ICML 2019 · similarity 0.88)
- Multi-objective Differentiable Neural Architecture Search (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
