Research map / Architectures and efficiency / Neural architecture search
Demystifying the Neural Tangent Kernel From a Practical Perspective: Can It Be Trusted for Neural Architecture Search Without Training?
Jisoo Mok, Byunggook Na, Ji-Hoon Kim, Dongyoon Han, Sungroh Yoon
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LGA after one training epoch instead of the best-performing prior at-initialization NTK metric | NAS architecture ranking · NAS-Bench-101 | Kendall's Tau: 0.094 -> 0.308 | Improved |
| Use LGA after one training epoch instead of the best-performing prior at-initialization NTK metric | NAS architecture ranking · NDS-DARTS | Kendall's Tau: 0.234 -> 0.408 | Improved |
| Use LGA after one training epoch instead of the best-performing prior at-initialization NTK metric | NAS architecture ranking · NDS-ENAS | Kendall's Tau: 0.162 -> 0.416 | Improved |
| Use LGA after one training epoch instead of the best-performing prior at-initialization NTK metric | NAS architecture ranking · NDS-NASNet | Kendall's Tau: 0.146 -> 0.357 | Improved |
| Use LGA after one training epoch instead of the best-performing prior at-initialization NTK metric | NAS architecture ranking · NDS-Amoeba | Kendall's Tau: 0.158 -> 0.396 | Improved |
| Use LGA₅ rather than LGA₃ | Random search on NAS-Bench-201 · CIFAR-10 | test accuracy: 93.64 -> 94.03 | Improved |
| Use LGA₅ rather than LGA₃ | Random search on NAS-Bench-201 · CIFAR-100 | test accuracy: 69.77 -> 71.56 | Improved |
| Use LGA₅ rather than LGA₃ | Random search on NAS-Bench-201 · ImageNet-16-120 | test accuracy: 45.03 -> 46.30 | Improved |
| Use LGA₅ rather than LGA₃ | Evolutionary search on NAS-Bench-201 · CIFAR-10 | test accuracy: 94.30 -> 93.94 | Hurt |
| Use LGA₅ rather than LGA₃ | Evolutionary search on NAS-Bench-201 · CIFAR-100 | test accuracy: 71.18 -> 72.42 | Improved |
| Use LGA₅ rather than LGA₃ | Evolutionary search on NAS-Bench-201 · ImageNet-16-120 | test accuracy: 45.30 -> 45.17 | No clear effect |
| Use Gaussian initialization instead of Xavier or Kaiming initialization (ablation) | F-Norm and Mean · NAS-Bench-201 | Kendall's Tau | Hurt |
Closest papers by meaning
- The Surprising Effectiveness of Infinite-Width NTKs for Characterizing and Improving Model Training (AAAI 2025 · similarity 0.89)
- Contrastive Neural Architecture Search With Neural Architecture Comparators (CVPR 2021 · similarity 0.89)
- Understanding and Accelerating Neural Architecture Search With Training-Free and Theory-Grounded Metrics (TPAMI 2023 · similarity 0.88)
- Speedy Performance Estimation for Neural Architecture Search (NeurIPS 2021 · similarity 0.88)
- Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective (ICLR 2021 · similarity 0.88)
- MOTE-NAS: Multi-Objective Training-based Estimate for Efficient Neural Architecture Search (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
