Research map / Architectures and efficiency / Neural architecture search

Building Optimal Neural Architectures using Interpretable Knowledge

CVPR 2024 · Architectures and efficiency › Neural architecture search · approach: Latency-aware architecture search · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoBuild, a GNN-based predictor trained with a hop-level SRCC ranking loss so embedding norms score architecture modules, which are then used to build or prune search spaces for classification, segmentation, and Stable Diffusion inpainting.

Problem

Neural architecture search is expensive because of the huge search space and the high cost of evaluating each candidate architecture; the paper asks whether one can learn to build good architectures directly from a small set of evaluated ones.

Approach

A GNN predictor is trained with an added differentiable Spearman rank-correlation loss at every hop level, so that subgraph embedding norms rank by ground-truth performance; these norms act as interpretable importance scores for operations, layers, and macro subgraphs, which are used to construct architectures or shrink the search space.

Key results

• On MobileNetV3 and ProxylessNAS, hop-wise test SRCC of predictors ranges roughly 0.73 to 0.99, showing magnitude-ranked embeddings correlate with accuracy and latency.
• AutoBuild-NAS yields superior accuracy-latency Pareto frontiers over Stage-Full, Layer-Full and Layer-Insight across all MBv3 tests and PN-GPU, and reaches above 79% accuracy at under 5ms on MBv3-NPU.
• On SDv1.4 inpainting, the top-4 AutoBuild architectures reach average FID 10.13 versus 10.82 for exhaustive search over 800k architectures and 22.13 for the 68 evaluated architectures; best FID 9.96 vs 10.29.

Measured effects reported in the paper

Surprising findings

• On PN-CPU, only two of four reduced search spaces outperform the dataset Pareto frontier, showing construction is not uniformly successful.
• Layer-Insight gains over Layer-Full were quite small and inconsistent, suggesting manual reduction from prior insights helps little.

Limitations

• Target equations used to combine accuracy and latency are manually inferred from data rather than automated.
• Relies on evaluated architectures, and on the SD task only 68 labeled architectures are used, with each fine-tune taking 1-2 days and 60GB VRAM.

Experimental setup

Datasets: ImageNet MS-COCO Places365 AIO-P performance annotations OFA latency measurements from [25]

Benchmarks: ImageNet (OFA ProxylessNAS and MobileNetV3 search spaces) MS-COCO panoptic segmentation Places365 inpainting with SDv1.4 (FID)

Metrics: SRCC accuracy latency PQ FLOPs FID

Baselines: Stage-Full Layer-Full Layer-Insight Exhaustive Search with ensemble predictors AIO-P fine-tuned architectures Evaluated 68 architectures (SDv1.4 variants)

Builds on: Once-for-All SPMiner ordered embedding space differentiable SRCC (Blondel et al. 2020) SnapFusion

Closest papers by meaning

• Understanding and Simplifying One-Shot Architecture Search (ICML 2018 · similarity 0.89)
• Hierarchical Representations for Efficient Architecture Search (ICLR 2018 · similarity 0.89)
• Prior Knowledge Guided Neural Architecture Generation (ICML 2025 · similarity 0.88)
• Insights from the Use of Previously Unseen Neural Architecture Search Datasets (CVPR 2024 · similarity 0.87)
• Neural Architecture Retrieval (ICLR 2024 · similarity 0.86)
• Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach (CVPR 2024 · similarity 0.86)