Research map / Architectures and efficiency / Neural architecture search
Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Aux-NAS, an asymmetric soft-parameter-sharing architecture where NAS with l1 pruning of auxiliary-to-primary links lets auxiliary labels improve the primary task while keeping single-task inference cost.
Problem
Auxiliary learning: exploit auxiliary labels from an independent task to boost a primary task while keeping single-task inference cost, avoiding the negative transfer of shared-parameter multi-task methods.
Approach
Use separate task branches with asymmetric cross-task links: train with bi-directional connections in a NAS search space, apply l1 regularization to auxiliary-to-primary architecture weights so they vanish, then drop all cross-task links and the auxiliary branch at inference, so the primary network is a single-task network.
Key results
- On NYU v2 semantic segmentation (VGG-16, normal as auxiliary), Aux-NAS reaches 36.0 mIoU vs 33.5 for Single and 35.0 for AdaShare-Aux.
- On CityScapes segmentation with disparity auxiliary, Aux-NAS achieves 71.1 mIoU vs 68.3 for Single.
- Aux-NAS keeps single-task inference cost, and training time scales linearly (0.337s to 0.556s per batch from 1 to 2 auxiliary tasks).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add NAS-based pruning of aux-to-primary links to gradient-only method (ablation) | Aux-G (gradient only) · NYU v2 Seg, VGG-16 | mIoU: 35.4 -> 35.7 | Helped |
| add auxiliary features on top of gradients with NAS (Aux-NAS) (ablation) | Aux-NAS vs gradient+NAS · NYU v2 Seg, VGG-16 | mIoU: 35.7 -> 36.0 | Helped |
| bi-directional NAS-based asymmetric method (Aux-NAS) vs. optimization-based aux learning on shared Aux-Head | Aux-Head · NYU v2 Seg, VGG-16 | mIoU: 34.7 -> 36.0 (best opt-based 35.3) | Helped |
| combine Aux-NAS with optimization-based methods (e.g., GCS, PCGrad-Aux) | Aux-NAS · NYU v2 Seg, VGG-16 | mIoU: 36.0 -> 36.3 (with GCS) | Helped |
| Aux-NAS replaces Adashare-Aux as the aux-learning architecture | Aux-NAS · CityScapes Seg, VGG-16 | mIoU: 70.1 -> 71.1 (vs Adashare-Aux) | Helped |
| add NAS to gradient-only approach (Aux-G) on ResNet-50 seg | Aux-G-Layer · NYU v2 Seg, ResNet-50 | mIoU: Aux-G-Layer 32.9 vs Single 34.1 | No clear effect |
| bi-directional NAS on more auxiliary tasks (1 -> 2) | Aux-NAS · NYU v2 Normal, ViTBase | Mean error: 12.5 -> 12.2 | Helped |
Surprising findings
- Aux-Head with optimization-based methods like DWA or PCGrad gives only marginal gains (about 0.3-0.5 mIoU), while the architecture-based variants without any loss manipulation outperform them.
Limitations
- Auxiliary-to-primary weights are manually zeroed before evaluation, relying on convergence to near-zero values.
- Fusion operation and search space are fixed by design choices (e.g., 3-layer window) rather than searched; the authors note chasing per-layer fusion ops is out of scope.
Experimental setup
Datasets: NYU v2 CityScapes Taskonomy
Benchmarks: NYU v2 CityScapes Taskonomy
Metrics: mIoU pixel accuracy surface normal mean/median/RMSE error within-angle percentage Top-1 accuracy Top-5 accuracy
Baselines: Single Aux-Head AdaShare AdaShare-Aux Uncertainty weighting DWA PCGrad CAGrad GCS
Builds on: DARTS NDDR AdaShare DPT PCGrad
Closest papers by meaning
- TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search (CVPR 2021 · similarity 0.88)
- Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search (CVPR 2022 · similarity 0.87)
- Generic Neural Architecture Search via Regression (NeurIPS 2021 · similarity 0.87)
- AUXILIARY TASK UPDATE DECOMPOSITION: THE GOOD, THE BAD AND THE NEUTRAL (ICLR 2021 · similarity 0.87)
- MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning (CVPR 2020 · similarity 0.87)
- einspace: Searching for Neural Architectures from Fundamental Operations (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.