AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Network Expansion for Practical Training Acceleration

Ning Ding, Yehui Tang, Kai Han, Chao Xu, Yunhe Wang

CVPR 2023 · Architectures and efficiency › Pruning and efficient tuning · approach: Joint pruning and quantization · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Train a narrower ResNet32 first, then progressively add channelsResNet32 · CIFAR10Top1 Acc.(%): 94.41 -> 94.36No clear effect
Train a narrower ResNet56 first, then progressively add channelsResNet56 · CIFAR100Top1 Acc.(%): 75.80 -> 75.76No clear effect
Train a narrower network first, then progressively add channelsResNet101 · ImageNet-1kTop1 Acc.(%): 77.4 -> 77.2No clear effect
Train a shallow model and expand depth using EMA-model layersDeiT-base · ImageNet-1kTop1 Acc.(%): 81.8 -> 81.5No clear effect
Expand the ResNet101 backbone from a narrower starting widthDeepLabV3+ with ResNet101 · CityscapesmIoU(%): 76.9 -> 75.1Hurt
Set α0 = 0.5 and ng = 2 instead of α0 = 0.5 and ng = 3 (ablation)DeepLabV3+ with ResNet101 · CityscapesmIoU(%): 75.1 -> 76.4Improved
Increase the initial width and reduce the number of expansion stages (ablation)ResNet56 · CIFAR100Top1 Acc.: 75.76 -> 75.91; 1.45× -> 1.35×Mixed
Reduce the starting width to α0 = 0.125 (ablation)ResNet56 · CIFAR100Top1 Acc.: 75.76 -> 75.31Hurt
Use α0 = 1/3 and expand in 3 stages rather than α0 = 0.5 and 2 stages (ablation)DeiT-base · ImageNet-1kTop1 Acc.: 81.5 -> 80.2Hurt
Initialize added filters with orthogonal weights instead of random weights (ablation)ResNet56 · CIFAR100Top1 Acc.: 75.39 -> 75.76Improved
Initialize added filters with orthogonal weights instead of replicated old filters (ablation)ResNet56 · CIFAR100Top1 Acc.: 74.67 -> 75.76Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy