AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

NASOA: Towards Faster Task-Oriented Online Fine-Tuning With a Zoo of Models

Hang Xu, Ning Kang, Gengwei Zhang, Chuanlong Xie, Xiaodan Liang, Zhenguo Li

ICCV 2021 · Architectures and efficiency › Pruning and efficient tuning · approach: Adapter-based parameter-efficient tuning · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the NAS-searched ET-NAS model zoo to BOHB (ablation)BOHB fine-tuning · average over test datasetsaccuracy: 77.17% to 87.45%Improved
replace BOHB with online adaptive scheduler (ablation)NASOA · average over test datasetscompute cost: x40 -> x1Improved
use adaptive-depth MLP with hedge backprop instead of fixed MLP (L=10) (ablation)accuracy predictor · online error rateMAE: 8.45% -> 7.51%Improved
add online adaptation instead of offline fixed MLP predictor (ablation)NASOA · average over test datasetsaccuracy: +2.17%Improved
add NAS model zoo to online scheduler (vs OA only on existing models) (ablation)NASOA · average over test datasetsaccuracy: 83.7% -> 85.8%Improved
fine-tune pretrained model instead of training from scratch (ablation)ResNet18 · Flowers102accuracy: 94.4% -> 98.3%Improved
fine-tune pretrained model instead of training from scratch (ablation)ResNet18 · CUB-Birdsaccuracy: 51.6% -> 69.2%Improved
tune learning rate and frozen stage per dataset (ablation)ResNet50 fine-tuning · Flowers102 and Stanford-Caroptimal LR: 0.01 vs 0.1Mixed
freeze BN statistics during fine-tuning (ablation)fine-tuning · multiple datasetsaccuracyHurt
use larger model under short time budget (ablation)fine-tuning · multiple datasetsaccuracyHurt
use ET-NAS-G instead of RegNetY-16GFtraining step time · ImageNettraining step time (ms): 677.68 -> 133.97Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy