Research map / Optimization / Optimizers
EG-NAS: Neural Architecture Search with Fast Evolutionary Exploration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EG-NAS, which combines gradient descent for network weights with a CMA-ES evolutionary strategy for architecture parameters, using a compound fitness (accuracy plus similarity) to search DARTS and NAS-Bench-201 spaces at very low cost.
Problem
DARTS-style gradient-based NAS converges prematurely to local optima, while evolutionary NAS methods are too costly; the paper seeks global exploration of architectures at low search cost.
Approach
Update network weights by gradient descent but update architecture parameters alpha via CMA-ES-based evolution sampling several search directions, with a fitness function that rewards performance and penalizes or rewards cosine similarity to the current architecture depending on whether the new candidate beats the current one.
Key results
- EG-NAS reaches 2.53% test error on CIFAR-10 with 0.1 GPU-Days search cost and 3.2M params (Table 1).
- Searching directly on ImageNet gives 24.9% top-1 test error at 0.1 GPU-Days in the table (2.2 GPU-Days for the transferred variant at 25.6%).
- On NAS-Bench-201, EG-NAS achieves 93.56%, 70.91% and 46.13% test accuracy on CIFAR-10, CIFAR-100 and ImageNet16-120.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the compound fitness function with both performance loss L1 and similarity term L2 instead of either term alone (ablation) | EG-NAS evolutionary strategy · CIFAR-10 search phase | valid accuracy | Helped |
| Use L2 alone, without performance loss L1 (ablation) | EG-NAS evolutionary strategy · CIFAR-10 search phase | valid accuracy | Hurt |
| Set ζ=0.8 rather than ζ=1.0, with η=0.4 (ablation) | EG-NAS fitness function · CIFAR-10 search phase | valid accuracy: 85.53 -> 84.82 | Hurt |
| Set ζ=0.4 rather than ζ=1.0, with η=0.4 (ablation) | EG-NAS fitness function · CIFAR-10 search phase | valid accuracy: 85.53 -> 82.81 | Hurt |
| Set ζ=0.2 rather than ζ=1.0, with η=0.4 (ablation) | EG-NAS fitness function · CIFAR-10 search phase | valid accuracy: 85.53 -> 81.54 | Hurt |
| Set η=0.8 rather than η=0.4, with ζ=1.0 (ablation) | EG-NAS fitness function · CIFAR-10 search phase | valid accuracy: 85.53 -> 85.42 | No clear effect |
| Set η=1.0 rather than η=0.4, with ζ=1.0 (ablation) | EG-NAS fitness function · CIFAR-10 search phase | valid accuracy: 85.53 -> 84.36 | No clear effect |
| Increase population size λ (ablation) | EG-NAS evolutionary strategy · CIFAR-10 search phase | valid accuracy | Mixed |
| Search on ImageNet rather than CIFAR-10 | EG-NAS · ImageNet | top-1 test error (%): 24.9 -> 25.6 | Hurt |
Surprising findings
- DARTS- outperforms EG-NAS on NAS-Bench-201 CIFAR-10 and CIFAR-100 validation/test results, which the authors acknowledge.
Limitations
- Gains over the strongest gradient-based baselines (ProxylessNAS, P-DARTS, β-DARTS) on CIFAR-10 are not claimed; method is described as trading off against them.
- Search cost scales with N and λ (time complexity N times λ relative to gradient methods), and population size beyond 50 yields marginal gains.
- Sensitivity to fitness coefficients ζ and η requires tuning.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet NAS-Bench-201
Benchmarks: CIFAR-10 CIFAR-100 ImageNet NAS-Bench-201
Metrics: test error top-1 accuracy search cost (GPU-Days) parameters (M)
Baselines: DARTS PC-DARTS P-DARTS β-DARTS DARTS- DrNAS DARTS+PT ProxylessNAS AmoebaNet NSGA-Net EAEPSO EPCNAS
Builds on: DARTS CMA-ES PC-DARTS
Closest papers by meaning
- iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · similarity 0.93)
- Rethinking Architecture Selection in Differentiable NAS (ICLR 2021 · similarity 0.92)
- Understanding Architectures Learnt by Cell-based Neural Architecture Search (ICLR 2020 · similarity 0.92)
- Neural Architecture Generator Optimization (NeurIPS 2020 · similarity 0.91)
- BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule (CVPR 2022 · similarity 0.91)
- Runtime Analysis of Evolutionary NAS for Multiclass Classification (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.