Research map / Architectures and efficiency / Neural architecture search
Understanding and Robustifying Differentiable Architecture Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Stop the search when the smoothed dominant architecture-Hessian eigenvalue rises, returning an earlier architecture | DARTS · CIFAR-10, CIFAR-100, and SVHN; search spaces S1-S4 | test error: CIFAR-10 S1: 4.66 ± 0.71 -> 3.05 ± 0.07; CIFAR-10 S2: 4.42 ± 0.40 -> 3.41 ± 0.14; CIFAR-10 S3: 4.12 ± 0.85 -> 3.71 ± 1.1 | Helped |
| Use stronger L2 regularization during architecture search (ablation) | DARTS · Image classification search spaces S1-S4 | test error | Helped |
| Apply Cutout and ScheduledDropPath during architecture search (ablation) | DARTS · Image classification search spaces S1-S4 | test error | Helped |
| Increase data augmentation strength during architecture search (ablation) | DARTS · FlyingThings3D and Sintel disparity estimation | EPE: FT test: 3.83 -> 2.33; Sintel test: 5.69 -> 3.76 | Mixed |
| Increase L2 regularization during architecture search (ablation) | DARTS · FlyingThings3D and Sintel disparity estimation | EPE: FT test: 3.25, 2.30, 2.72, 2.34; Sintel test: 6.13, 4.12, 4.83, 3.85 | Mixed |
| Use L2 regularization stronger than the default in the search (ablation) | DARTS · Penn Treebank | test perplexity | Helped |
| Adaptively increase regularization when the eigenvalue-based stopping criterion is triggered | DARTS · Image classification search spaces S1-S4 | test error | Helped |
| Run architecture search at multiple regularization strengths and select by validation performance | DARTS · CIFAR-100, SVHN, and Penn Treebank | test error or perplexity: CIFAR-100: 20.58 ± 0.44 -> 18.01 ± 0.26; SVHN: 2.46 ± 0.09 -> 2.17 ± 0.09; Penn Treebank: 58.64 -> 57.59 | Helped |
| Run architecture search at multiple regularization strengths and select by validation performance | DARTS · CIFAR-10 | test error: 2.91 ± 0.25 -> 2.95 ± 0.21 | No clear effect |
| Increase the ScheduledDropPath probability during search (ablation) | DARTS · CIFAR-10 S1-S4 | test accuracy: S1 seed 1: 96.16 -> 96.89; S2 seed 1: 95.15 -> 96.52; S3 seed 1: 94.70 -> 96.91; S4 seed 1: 92.80 -> 96.42 | Mixed |
| Increase L2 regularization too far during architecture search (ablation) | DARTS · CIFAR-10 S5 | test error | Hurt |
Closest papers by meaning
- Stabilizing Differentiable Architecture Search via Perturbation-based Regularization (ICML 2020 · similarity 0.93)
- DARTS: Differentiable Architecture Search (ICLR 2019 · similarity 0.91)
- Differentiable Architecture Search With Random Features (CVPR 2023 · similarity 0.90)
- DARTS-: Robustly Stepping out of Performance Collapse Without Indicators (ICLR 2021 · similarity 0.89)
- iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · similarity 0.89)
- BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule (CVPR 2022 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.