AI Paper Atlas

Research map / Architectures and efficiency / Neural architecture search

NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size

Xuanyi Dong, Lu Liu, Katarzyna Musiał, Bogdan Gabryś

TPAMI 2021 · Architectures and efficiency › Neural architecture search · approach: Efficient NAS search strategies · 190 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use REA instead of random searchmulti-trial NAS · CIFAR-100 (topology space)test accuracy: 71.55 -> 72.23Improved
use REA instead of random searchmulti-trial NAS · ImageNet-16-120 (size space)test accuracy: 45.42 -> 45.94Improved
use second-order instead of first-order DARTSDARTS · CIFAR-10 (topology space)test accuracy: 59.84 -> 65.38Improved
use second-order instead of first-order DARTSDARTS · ImageNet-16-120 (topology space)test accuracy: 37.88 -> 36.79Hurt
search on CIFAR-100 instead of CIFAR-10DARTS · topology spacetest accuracy: 59.84 -> 61.26Improved
sample candidates with shared weights and select best validation instead of argmax over architecture parametersweight-sharing NAS (RSPS/ENAS/SETN) · CIFAR-10 (topology space)test accuracy: 59.84 -> 93.76Improved
use channel-wise interpolation instead of masking + samplingsize-space weight-sharing search · ImageNet-16-120 (size space)test accuracy: 45.11 -> 47.17Improved
use channel-wise interpolation instead of masking + samplingsize-space weight-sharing search · CIFAR-10 (size space)test accuracy: 92.78 -> 93.40Improved
use 200 epochs instead of 12 epochs for trial feedback (ablation)REA · CIFAR-10 (topology space)test accuracy of discovered architecture: after 1.2e6 seconds still worse than 12 epochsHurt
use 90 epochs instead of 12 epochs for trial feedback (ablation)REA · CIFAR-10 (size space)test accuracy of discovered architecture: outperforms after about 8e4 secondsMixed
use weight sharing instead of multi-trial searchNAS algorithms · CIFAR-100 (topology space)test accuracy: 72.23 -> 68.26Hurt
use REA search instead of largest candidatesize space · ImageNet-16-120test accuracy: 47.17 -> 45.94Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy