Research map / Architectures and efficiency / Neural architecture search
BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model architecture parameters as a Gaussian distribution and optimize it with the Bayesian learning rule | DARTS · NAS-Bench-201 | test accuracy: DARTS (2nd): 54.30 -> BaLeNAS (2nd): 94.02; 15.61 -> 71.93; 16.32 -> 45.48 | Helped |
| Select sampled architectures with training-free Synflow instead of taking argmax of the distribution mean (ablation) | BaLeNAS · NAS-Bench-201 | test accuracy: 94.02 -> 94.33; 71.93 -> 72.95; 45.48 -> 46.54 | Helped |
| Increase Synflow-selected architecture sample size from 10 to 50 (ablation) | BaLeNAS-TF · NAS-Bench-201 | test accuracy: 94.08 -> 94.33; 72.55 -> 72.95; 45.82 -> 46.54 | Helped |
| Increase Synflow-selected architecture sample size from 50 to 100 (ablation) | BaLeNAS-TF · NAS-Bench-201 | test accuracy: 94.33 -> 94.33; 72.95 -> 72.95; 46.54 -> 46.54 | No clear effect |
| Use SNIP rather than Synflow to select sampled architectures (ablation) | BaLeNAS-TF · NAS-Bench-201 | test accuracy: CIFAR-10: 94.02 -> 93.85; CIFAR-100: 71.93 -> 71.48; ImageNet-16-120: 45.48 -> 46.03 | Mixed |
| Use GraSP rather than Synflow to select sampled architectures (ablation) | BaLeNAS-TF · NAS-Bench-201 | test accuracy: CIFAR-10: 94.02 -> 93.94; CIFAR-100: 71.93 -> 72.00; ImageNet-16-120: 45.48 -> 44.67 | Mixed |
| Increase MCMC sample count from 1 to 3 (ablation) | BaLeNAS (2nd) · NAS-Bench-201 CIFAR-10 | test accuracy: 93.33 -> 94.02 | Helped |
| Increase MCMC sample count from 3 to 4 (ablation) | BaLeNAS (2nd) · NAS-Bench-201 | test accuracy: CIFAR-10: 94.02 -> 93.04; CIFAR-100: 71.93 -> 69.20; ImageNet-16-120: 45.48 -> 43.21 | Hurt |
| Use BaLeNAS-TF rather than BaLeNAS for architecture selection (ablation) | DARTS search space · CIFAR-10 test | test error: 2.50 -> 2.43 | Helped |
| Use BaLeNAS-TF rather than BaLeNAS for architecture selection (ablation) | DARTS search space · CIFAR-100 test | test error: 16.84 -> 15.72 | Helped |
Closest papers by meaning
- iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · similarity 0.94)
- Rethinking Architecture Selection in Differentiable NAS (ICLR 2021 · similarity 0.93)
- UNAS: Differentiable Architecture Search Meets Reinforcement Learning (CVPR 2020 · similarity 0.92)
- Differentiable Architecture Search With Random Features (CVPR 2023 · similarity 0.92)
- EG-NAS: Neural Architecture Search with Fast Evolutionary Exploration (AAAI 2024 · similarity 0.91)
- DrNAS: Dirichlet Neural Architecture Search (ICLR 2021 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.