AI Paper Atlas

Research map / Architectures and efficiency / Neural architecture search

Analyzing and Mitigating Interference in Neural Architecture Search

Jin Xu, Xu Tan, Kaitao Song, Renqian Luo, Yichong Leng, Tao Qin, Tie-Yan Liu, Jian Li

ICML 2022 · Architectures and efficiency › Neural architecture search · approach: Bi-level differentiable NAS · 11 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Gradually substitute one operator between adjacent training stepsSPOS training in the BERT search space · BERT search space, 60 sampled child modelsKendall's Tau: 0.36 -> 0.49Improved
Add anchor-based input/output alignmentSPOS training in the BERT search space · BERT search space, 60 sampled child modelsKendall's Tau: 0.36 -> 0.44Improved
Combine gradual topology changes with anchor alignmentSPOS training in the BERT search space · BERT search space, 60 sampled child modelsKendall's Tau: 0.36 -> 0.57Improved
Use MAGIC-AT rather than SPOS to discover an MLM-pretrained architectureBERT-base-scale pre-training · GLUE devAVG: 84.4 -> 85.7Improved
Use MAGIC-AT rather than SPOS to discover an MLM-pretrained architectureBERT-base-scale pre-training · GLUE testAVG: 82.4 -> 83.4Improved
Use MAGIC-AT rather than SPOS to discover a compressed architecture60M distilled BERT · GLUE devAVG: 83.1 -> 84.8Improved
Use MAGIC-AT rather than SPOS to discover a compressed architecture60M distilled BERT · GLUE testAVG: 81.4 -> 82.0Improved
Use MAGIC-AT rather than PC-NAS for architecture searchMobileNetV2 search space · ImageNetTop1 test error: 23.9 -> 23.2Improved
Increase the number of operators changed at each MAGIC-T step (ablation)MAGIC-T super-net training · BERT search spaceRank correlationHurt
Choose a poorly performing child model as the MAGIC-A anchor (ablation)MAGIC-A super-net training · BERT search spaceRank correlationHurt
Use an anchor model ranking within the top 30% rather than requiring the top-ranked model (ablation)MAGIC-A super-net training · BERT search spaceRank correlationNo clear effect
Align child-model inputs and outputs to the average across sampled paths (ablation)Shared operators in the analyzed hybrid BERT search space · Gradient similarity analysisGradient cosine similarityImproved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy