Research map / Architectures and efficiency / Neural architecture search
Evaluating Efficient Performance Estimators of Neural Architectures
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OS loss rather than OS accuracy to rank architectures (ablation) | One-shot estimator · NAS-Bench-301, epoch 1000 | KD τ: 0.381 -> 0.512 | Helped |
| Use OS loss rather than OS accuracy to rank architectures (ablation) | One-shot estimator · NAS-Bench-301, epoch 1000 | P@top5%: 13.8% -> 31.0% | Helped |
| Evaluate OS scores on more validation batches after sufficient training (ablation) | One-shot estimator · NAS-Bench-201 and NAS-Bench-301 | ranking quality: KD≈0.5 V.S. KD≈0.76 | Helped |
| Use more validation batches while the supernet is under-trained (ablation) | One-shot estimator · NAS-Bench-301, epoch 200 | ranking quality | Hurt |
| Use OS loss instead of OS accuracy (ablation) | One-shot estimator · NAS-Bench-101 and NAS-Bench-201 | ranking quality | No clear effect |
| Use multiple Monte Carlo architecture samples per training step (ablation) | One-shot estimator · NAS-Bench-101 and NAS-Bench-301 | KD τ: 0.384 -> 0.446; 0.530 -> 0.548 | Helped |
| Use multiple Monte Carlo architecture samples per training step (ablation) | One-shot estimator · NAS-Bench-201 | KD τ: 0.744 -> 0.714 | Hurt |
| Use de-isomorphism sampling to sample unique structures uniformly (ablation) | One-shot estimator · NAS-Bench-201 | BR@0.5%: 1.9% to 0.23% | Helped |
| Use de-isomorphism sampling to sample unique structures uniformly (ablation) | One-shot estimator · NAS-Bench-201 | P@5%: 21.3% to 46.7% | Helped |
| Average scores for architectures in an isomorphic group during testing (ablation) | One-shot estimator · NAS-Bench-201 | BR@K and P@topK | Helped |
| Prune the search space to the top-ranked architectures and continue supernet training (ablation) | One-shot estimator · NAS-Bench-201 | KD: 0.189/0.046/0.086 | Helped |
| Reduce parameter sharing by removing operations from the search space (ablation) | One-shot estimator · Search spaces studied | ranking quality | Mixed |
Closest papers by meaning
- Speedy Performance Estimation for Neural Architecture Search (NeurIPS 2021 · similarity 0.92)
- Rethinking Performance Estimation in Neural Architecture Search (CVPR 2020 · similarity 0.90)
- Few-Shot Neural Architecture Search (ICML 2021 · similarity 0.90)
- Generalizing Few-Shot NAS with Gradient Matching (ICLR 2022 · similarity 0.89)
- Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · similarity 0.89)
- MOTE-NAS: Multi-Objective Training-based Estimate for Efficient Neural Architecture Search (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.