Research map / Architectures and efficiency / Neural architecture search

HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning

AAAI 2025 · Architectures and efficiency › Neural architecture search · approach: One-shot and few-shot supernets · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HEP-NAS, a few-shot NAS method that partitions all edges of each intermediate node as one hierarchy, progressively selects the best sub-supernet, and uses search space mutual distillation to improve sub-supernet evaluation.

Problem

Few-shot NAS edge-wise supernet partitioning ignores interrelationships among edges sharing an end node, leaving co-adaptation within hierarchies and wasting search budget on large search spaces.

Approach

Treat edges sharing the same end node as a hierarchy and split all of them jointly into Cartesian-product operation-group combinations, keep only the best sub-supernet after each split, and train sub-supernets with mutual distillation among siblings plus guidance from the previous optimal sub-supernet.

Key results

• On DARTS search space CIFAR-10, HEP-NAS reaches 2.44% top-1 error with 3.6M params and 1.5 GPU-days, versus 2.60% for few-shot DARTS and 2.46% for GM DARTS.
• On CIFAR-100 in the DARTS space, HEP-NAS achieves 16.83% error, the best among compared methods.
• On NAS-Bench-201, HEP-NAS reports 46.51% average test accuracy on ImageNet16-120 and 93.86% on CIFAR-10, with a reported Spearman correlation of 0.665.

Measured effects reported in the paper

Surprising findings

• Splitting hierarchies in reverse order or random order does not hurt much: random order gives 2.63% CIFAR-10 error versus 2.44% for the proposed order, indicating the ordering effect is modest.
• Splitting only one hierarchy yields 17.2% CIFAR-100 error at 0.38 GPU-days, close to full splitting at 16.83%.

Limitations

• Search cost is somewhat higher than original few-shot methods because the best sub-supernet must be selected after every split.
• Reported improvements are often small (e.g., 2.44% vs 2.46%), and results on NAS-Bench-201 are not always best across all datasets.
• Hierarchy-wise partitioning produces many sub-supernets per stage, which grows with the number of incoming edges.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 ImageNet ImageNet16-120 NAS-Bench-201

Benchmarks: CIFAR-10 CIFAR-100 ImageNet NAS-Bench-201 (CIFAR-10, CIFAR-100, ImageNet16-120)

Metrics: top-1 error rate top-1 accuracy search cost (GPU-days) params Spearman correlation

Baselines: ResNet DenseNet ProxylessNAS PC-DARTS P-DARTS DARTS- DARTS-PT DDPNAS ADARTS IS-DARTS MOEA MixPath

Builds on: gradient matching NAS (GM-NAS) DARTS few-shot NAS NAS-Bench-201

Closest papers by meaning

• Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · similarity 0.94)
• Few-Shot Neural Architecture Search (ICML 2021 · similarity 0.93)
• Generalizing Few-Shot NAS with Gradient Matching (ICLR 2022 · similarity 0.92)
• Subnet-Aware Dynamic Supernet Training for Neural Architecture Search (CVPR 2025 · similarity 0.91)
• Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach (CVPR 2024 · similarity 0.90)
• Revisiting Neural Networks for Few-Shot Learning: A Zero-Cost NAS Perspective (ICML 2025 · similarity 0.89)