Research map / Architectures and efficiency / Neural architecture search
Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes semi-automatically structured design knowledge (an LLM-populated attribute tree over 1,149 CVPR 2023 papers) plus FairNAD, a multi-type-mutation LLM-driven open-ended NAS, reaching state-of-the-art NAS-Bench-201 accuracy.
Problem
Open-ended LLM-driven NAS explores inefficiently because design ideas drawn from papers and LLM priors are unstructured, biased toward research trends, and of mixed quality and scope.
Approach
Structure design knowledge with a three-level attribute tree (granularity, main category, sub-category) built from reference SOTA model code and a predefined template, tag paper-derived ideas with it, and sample ideas uniformly over attributes within a multi-type mutation search (fair idea sampling, Pareto-aware mutation, LLM iterative mutation) guarded by a feedback loop.
Key results
- FairNAD (500 archs) improves test accuracy over NADER by 0.84 (CIFAR-10, 95.46 vs 94.62), 2.17 (CIFAR-100, 78.17 vs 76.00), and 2.35 (ImageNet16-120, 52.87 vs 50.52) points.
- Structured inspiration adds 0.37 points over paper-based ideas from NADER in the idea-based mutation ablation (200 architectures, CIFAR-100).
- Removing the budget verifier drops the pass rate from 0.56 to 0.04 and CIFAR-100 test accuracy by 1.34 points.
Surprising findings
- Applying the LLM-driven iterative mutation stage alone degraded CIFAR-100 test accuracy by 1.41 points, becoming beneficial only when combined with Pareto-aware mutation.
- Adding sub-category attributes for fair sampling degraded test accuracy by 2.18 points relative to main-category sampling, attributed to overly fragmented groups.
Limitations
- Attribute tree sub-categories suffer from LLM inconsistency and over-specific attributes, requiring manual correction.
- Results depend on the coding ability of the LLM; smaller LLMs yield lower-performing searches.
- Evaluated only on NAS-Bench-201 with vision backbones; the design-knowledge source is limited to CVPR 2023 papers and reference models.
Experimental setup
Datasets: CVPR 2023 open-access papers NAS-Bench-201 CIFAR-10 NAS-Bench-201 CIFAR-100 NAS-Bench-201 ImageNet16-120
Benchmarks: NAS-Bench-201 CIFAR-10 NAS-Bench-201 CIFAR-100 NAS-Bench-201 ImageNet16-120
Metrics: validation accuracy test accuracy pass rate parameters FLOPs
Baselines: NADER LLMatic GENIUS CoLLM-NAS LeMo-NADe REA DARTS DRNAS
Builds on: NADER NSGA-II LLMatic GENIUS
Closest papers by meaning
- RZ-NAS: Enhancing LLM-guided Neural Architecture Search via Reflective Zero-Cost Strategy (ICML 2025 · similarity 0.91)
- Design Principle Transfer in Neural Architecture Search via Large Language Models (AAAI 2025 · similarity 0.90)
- Towards Neural Architecture Search through Hierarchical Generative Modeling (ICML 2024 · similarity 0.88)
- EG-NAS: Neural Architecture Search with Fast Evolutionary Exploration (AAAI 2024 · similarity 0.88)
- einspace: Searching for Neural Architectures from Fundamental Operations (NeurIPS 2024 · similarity 0.87)
- Insights from the Use of Previously Unseen Neural Architecture Search Datasets (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.