Research map / Architectures and efficiency / Neural architecture search

IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance

AAAI 2024 · Architectures and efficiency › Neural architecture search · approach: Bi-level differentiable NAS · 20 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IS-DARTS, a DARTS variant that ranks candidate operations by a Fisher-information-based importance measure (IIM) on validation feature maps and progressively shrinks comparison subsets in the supernet to fix performance collapse.

Problem

DARTS suffers from performance collapse, where architecture parameters fail to reflect operation importance and the selected architectures are dominated by parameter-free operations; bi-level optimization further wastes data and focuses on the supernet rather than subnets.

Approach

Replace architecture-parameter-based selection with an information-based importance measure (IIM) derived from Fisher information of intermediate feature maps, use binary masks with a validation-inclusive training objective, and progressively shrink operations per comparison subset to connect supernet and subnet.

Key results

• On NAS-Bench-201, IS-DARTS reaches 94.36% CIFAR-10 test, 73.51% CIFAR-100 test and 46.34% ImageNet-6-120 test in 2.0 GPU hours, averaged over 4 runs.
• On the DARTS space (CIFAR-10), IS-DARTS achieves 2.56±0.04% test error at 0.42 GPU days; the CIFAR-10-searched architecture gets 24.1% top-1 error on ImageNet.
• Ablation on NAS-Bench-201: removing the IIM selection (S-DARTS) collapses CIFAR-10 test accuracy to 70.92%, while removing the shrinking pipeline (I-DARTS) gives 93.31%.

Measured effects reported in the paper

Surprising findings

• Architecture parameters ranked operations in a way that did not match their measured evaluation accuracies: swapping higher-alpha operations for lower-alpha ones gave unordered subnet accuracies.
• Using only the shrinking pipeline without IIM-based selection (S-DARTS) collapsed to 70.92% CIFAR-10 test accuracy on NAS-Bench-201, showing the shrinking alone is insufficient.

Limitations

• Relies on a Gaussian independence assumption for feature map pixels to derive the IIM, which is an approximation.
• The one-step weight approximation Gamma_0 to Gamma assumes supernet weights remain near-optimal in subnets, which the paper acknowledges introduces errors.
• Searched architecture on DARTS space has larger parameter count (4.25M) than the other reported DARTS variants (3.5M).

Experimental setup

Datasets: CIFAR-10 CIFAR-100 ImageNet ImageNet-6-120

Benchmarks: NAS-Bench-201 DARTS search space CIFAR-10 DARTS search space ImageNet

Metrics: test accuracy test error search cost (GPU hours/days) parameters (M)

Baselines: DARTS ENAS GDAS SETN DSNAS PC-DARTS DARTS- FairDARTS SDARTS-ADV SGAS P-DARTS

Builds on: DARTS PC-DARTS PDARTS SGAS DARTS-

Closest papers by meaning

• $\Lambda$-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells (ICLR 2023 · similarity 0.92)
• iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · similarity 0.90)
• Subnet-Aware Dynamic Supernet Training for Neural Architecture Search (CVPR 2025 · similarity 0.87)
• Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach (CVPR 2024 · similarity 0.87)
• EG-NAS: Neural Architecture Search with Fast Evolutionary Exploration (AAAI 2024 · similarity 0.87)
• Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · similarity 0.85)