Research map / Architectures and efficiency / Neural architecture search
Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search
Xiangxiang Chu, Tianbao Zhou, Bo Zhang, Jixiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add zero-one loss (ablation) | FairDARTS · CIFAR-10 test, S1 | top-1 accuracy (%): 97.33 -> 97.41 | Improved |
| add zero-one loss (ablation) | FairDARTS · CIFAR-10 test, S1 | multiply-adds (M): 532 -> 458 | Improved |
| add zero-one loss to softmax DARTS (ablation) | DARTS · CIFAR-10 test, S1 | top-1 accuracy (%): 96.76 -> 96.77 | No clear effect |
| replace softmax with independent sigmoid (collaborative competition) (ablation) | DARTS · ImageNet S2 search | top-1 accuracy of derived model (%): 66.4 -> 75.6 | Improved |
| remove skip connections from search space (ablation) | DARTS · CIFAR-10 test, S1 | top-1 accuracy (%): 96.76 -> 96.88 | Improved |
| use FairDARTS instead of DARTS without skip connections | S1 · CIFAR-10 test | top-1 accuracy (%): 96.88 -> 97.41 | Improved |
| add Gaussian noise to skip-connection architecture weights (ablation) | DARTS · CIFAR-10 test, S1 | top-1 accuracy (%): 97.12 | Improved |
| restrict skip connections to M=2 and sample randomly (ablation) | random search · CIFAR-10 test, S1 | top-1 accuracy (%): 97.01 -> 97.14 | Improved |
| use FairDARTS-C with SE, Swish and AutoAugment instead of EfficientNet B0 | FairDARTS-C · ImageNet val | top-1 accuracy (%): 76.3 -> 77.2 | Improved |
| use FairDARTS-D instead of PC-DARTS | ImageNet S2 · ImageNet val | top-1 accuracy (%): 74.9 -> 75.6 | Improved |
| use FairDARTS-C backbone instead of MixNet-M | RetinaNet · COCO val2017 | AP (%): 31.3 -> 31.9 | Improved |
| raise sigma threshold from 0.75 to 0.8 (ablation) | FairDARTS · ImageNet val | params / accuracy: 75.6 -> 75.1 | Mixed |
Later papers that compared with this method
7 later accepted papers report FairDARTS as a baseline in their results tables; 7 of them report a better number on most of the results they share with it.
- Bridge the Gap Between Architecture Spaces via A Cross-Domain Predictor (NeurIPS 2022 · better on 1 of 1 shared result)
- Cyclic Differentiable Architecture Search (TPAMI 2022 · better on 1 of 1 shared result)
- Efficient Joint-Dimensional Search with Solution Space Regularization for Real-Time Semantic Segmentation (IJCV 2022 · better on 1 of 1 shared result)
- IDARTS: Interactive Differentiable Architecture Search (ICCV 2021 · better on 2 of 2 shared results)
- Sample-Efficient Neural Architecture Search by Learning Actions for Monte Carlo Tree Search (TPAMI 2021 · better on 2 of 2 shared results)
- EC-DARTS: Inducing Equalized and Consistent Optimization Into DARTS (ICCV 2021 · better on 1 of 1 shared result)
- Theory-Inspired Path-Regularized Differential Network Architecture Search (NeurIPS 2020 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients (ICML 2021 · similarity 0.92)
- $\Lambda$-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells (ICLR 2023 · similarity 0.92)
- EC-DARTS: Inducing Equalized and Consistent Optimization Into DARTS (ICCV 2021 · similarity 0.91)
- Cyclic Differentiable Architecture Search (TPAMI 2022 · similarity 0.90)
- DARTS-: Robustly Stepping out of Performance Collapse Without Indicators (ICLR 2021 · similarity 0.90)
- IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
