Research map / Architectures and efficiency / Neural architecture search
MOTE-NAS: Multi-Objective Training-based Estimate for Efficient Neural Architecture Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MOTE, a training-based zero/few-cost estimate combining a loss-landscape term and a unit-time training-speed term, computed on a reduced architecture and reduced CIFAR-100 subset, and embeds it in an evolutionary NAS (MOTE-NAS) that reaches new accuracy-cost trade-offs on NASBench-201.
Problem
Neural architecture search is expensive because candidate architectures must be trained to get accurate performance estimates; existing NTK-based training-free estimates ignore the non-convex, nonlinear training dynamics of finite-width DNNs.
Approach
MOTE sums a landscape term (cross-entropy along linear interpolations between initial and trained weights) and a speed term (per-epoch training loss divided by epoch time), transformed with box-cox, and uses reduced architecture and reduced dataset to make the estimate cheap (~7 GPU seconds per candidate), integrated into an iterative evolutionary search with top-k early-stopping evaluation.
Key results
- MOTE-NAS reaches 94.32% on CIFAR-10, 72.81% on CIFAR-100 and 46.38% on ImageNet-16-120 on NASBench-201 (k=20 / k=10), outperforming NTK-based NAS methods.
- MOTE achieves Kendall's tau of about 0.68 on NASBench-101 and 0.47 on NASBench-201 (CIFAR-100), with ~7 GPU seconds per candidate.
- MOTE-NAS-EF (evaluation-free) reaches 93.54%/71.59%/44.73% at about 0.6K GPU seconds, matching KNAS accuracy at 4.8x to 22.2x lower cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine the landscape and speed terms in MOTE for candidate ranking | NASBench-101 architectures · NASBench-101 | Kendall's Tau correlation: 0.47 | Helped |
| use MOTE-NAS instead of TE-NAS, KNAS, or Eigen-NAS | NASBench-201 search · NASBench-201 | accuracy: 94.32; 72.81; 46.38 | Helped |
| omit the evaluation stage and select the top-1 architecture using MOTE (ablation) | MOTE-NAS · NASBench-201 CIFAR-10 | accuracy: 94.32 -> 93.54 | Hurt |
| omit the evaluation stage and select the top-1 architecture using MOTE (ablation) | MOTE-NAS · NASBench-201 CIFAR-100 | accuracy: 72.81 -> 71.59 | Hurt |
| omit the evaluation stage and select the top-1 architecture using MOTE (ablation) | MOTE-NAS · NASBench-201 ImageNet-16-120 | accuracy: 46.38 -> 44.73 | Hurt |
| use box-cox rather than no transformation (ablation) | MOTE scoring · NASBench-201 CIFAR-10 | Kendall's τ correlation: 0.66 -> 0.68 | Helped |
| use box-cox rather than logarithm (ablation) | MOTE scoring · NASBench-201 CIFAR-100 | Kendall's τ correlation: 0.64 -> 0.66 | Helped |
| use box-cox rather than reciprocal (ablation) | MOTE scoring · NASBench-201 ImageNet-16-120 | Kendall's τ correlation: 0.60 -> 0.62 | Helped |
| replace the original meta-architecture with a reduced architecture and rank by early-stopping accuracy (ablation) | NASBench-101 candidate ranking · NASBench-101 | Kendall's τ correlation (%): 34.5 -> 8.1 | Hurt |
| use MOTE with the reduced architecture instead of early-stopping accuracy with the reduced architecture (ablation) | NASBench-101 candidate ranking · NASBench-101 | Kendall's τ correlation (%): 8.1 -> 50.1 | Helped |
| add a reduced dataset with r=10 labels to MOTE + RA (ablation) | NASBench-101 candidate ranking · NASBench-101 | Kendall's τ correlation (%): 50.1 -> 46.6 | Hurt |
| replace evolutionary search with random sampling (ablation) | MOTE-NAS · NASBench-201 | accuracy: 94.07%; 72.60%; 46.13% | Mixed |
Surprising findings
- Early-stopping test accuracy after 12 epochs loses correlation (to ~0.5 or below) under reduction strategies, whereas the MOTE terms keep high correlation under the same reductions.
Limitations
- MOTE-NAS does not comprehensively outperform all advanced methods (e.g., REA+LGA on CIFAR-10 and ImageNet-16 variants).
- Evaluated mainly on NASBench-101/201; generalization to NASBench-301 and other search spaces is untested.
- Evaluation-free variant incurs accuracy loss relative to the full MOTE-NAS.
Experimental setup
Datasets: NASBench-101 NASBench-201 CIFAR-100 CIFAR-10 ImageNet-16-120 ImageNet-1K
Benchmarks: NASBench-101 NASBench-201 ImageNet-1K
Metrics: test accuracy Kendall's tau search cost (GPU seconds)
Baselines: TE-NAS KNAS Eigen-NAS LGA NASWOT ZICO SynFlow TSE Neural Predictor WeakNAS Proxy-BO OFA
Builds on: NTK-based NAS (TE-NAS) Training Speed Estimation (TSE) KNAS Label-Gradient Alignment (LGA) NASBench-201
Closest papers by meaning
- Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective (ICLR 2021 · similarity 0.91)
- Speedy Performance Estimation for Neural Architecture Search (NeurIPS 2021 · similarity 0.90)
- Per-Architecture Training-Free Metric Optimization for Neural Architecture Search (NeurIPS 2025 · similarity 0.89)
- Evaluating Efficient Performance Estimators of Neural Architectures (NeurIPS 2021 · similarity 0.89)
- Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search (NeurIPS 2022 · similarity 0.89)
- Robustifying and Boosting Training-Free Neural Architecture Search (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.