Research map / Architectures and efficiency / Pruning and efficient tuning
NASOA: Towards Faster Task-Oriented Online Fine-Tuning With a Zoo of Models
Hang Xu, Ning Kang, Gengwei Zhang, Chuanlong Xie, Xiaodan Liang, Zhenguo Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the NAS-searched ET-NAS model zoo to BOHB (ablation) | BOHB fine-tuning · average over test datasets | accuracy: 77.17% to 87.45% | Improved |
| replace BOHB with online adaptive scheduler (ablation) | NASOA · average over test datasets | compute cost: x40 -> x1 | Improved |
| use adaptive-depth MLP with hedge backprop instead of fixed MLP (L=10) (ablation) | accuracy predictor · online error rate | MAE: 8.45% -> 7.51% | Improved |
| add online adaptation instead of offline fixed MLP predictor (ablation) | NASOA · average over test datasets | accuracy: +2.17% | Improved |
| add NAS model zoo to online scheduler (vs OA only on existing models) (ablation) | NASOA · average over test datasets | accuracy: 83.7% -> 85.8% | Improved |
| fine-tune pretrained model instead of training from scratch (ablation) | ResNet18 · Flowers102 | accuracy: 94.4% -> 98.3% | Improved |
| fine-tune pretrained model instead of training from scratch (ablation) | ResNet18 · CUB-Birds | accuracy: 51.6% -> 69.2% | Improved |
| tune learning rate and frozen stage per dataset (ablation) | ResNet50 fine-tuning · Flowers102 and Stanford-Car | optimal LR: 0.01 vs 0.1 | Mixed |
| freeze BN statistics during fine-tuning (ablation) | fine-tuning · multiple datasets | accuracy | Hurt |
| use larger model under short time budget (ablation) | fine-tuning · multiple datasets | accuracy | Hurt |
| use ET-NAS-G instead of RegNetY-16GF | training step time · ImageNet | training step time (ms): 677.68 -> 133.97 | Improved |
Closest papers by meaning
- MIGO-NAS: Towards Fast and Generalizable Neural Architecture Search (TPAMI 2021 · similarity 0.88)
- DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search (ECCV 2020 · similarity 0.87)
- Neural Fine-Tuning Search for Few-Shot Learning (ICLR 2024 · similarity 0.87)
- PASHA: Efficient HPO and NAS with Progressive Resource Allocation (ICLR 2023 · similarity 0.87)
- TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search (CVPR 2021 · similarity 0.86)
- Robustifying and Boosting Training-Free Neural Architecture Search (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
