AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Towards Adaptive Residual Network Training: A Neural-ODE Perspective

Chengyu Dong, Liyuan Liu, Zichao Li, Jingbo Shang

ICML 2020 · Architectures and efficiency › RNNs and normalization · approach: Efficient network compression and training

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Adaptive depth growth from shallow start (LipGrow) instead of training a deep ResNet from scratchResNet-50 and ResNet-74 on CIFAR-10 · CIFAR-10Test accuracy: -0.60 (ResNet-50), -0.30 (ResNet-74)Mixed
Adaptive depth growth from shallow start (LipGrow)ResNet-50 on CIFAR-100 · CIFAR-100Test accuracy: +0.30Improved
Adaptive depth growth from shallow start (LipGrow)ResNet-74 on CIFAR-100 · CIFAR-100Test accuracy: -0.41Hurt
Adaptive depth growth from shallow start (LipGrow)ResNet-66 on Tiny-ImageNet · Tiny-ImageNetTest accuracy: +0.69Improved
Adaptive depth growth from shallow start (LipGrow) reducing parameters per epochResNet on CIFAR and Tiny-ImageNet · CIFAR-10/100 and Tiny-ImageNetParameters per epoch (training compute proxy): roughly halved (about 50% reduction claimed)Improved
Clone-based initialization with implicit step-size scaling plus Lipschitz-triggered growth vs. hand-tuned schedule (ablation)LipGrow vs Hand-Tuned growing schedule · CIFAR-10Validation accuracy vs PPEMixed
Choose first growth epoch via Lipschitz-based LipGrow rather than grid search (ablation)ResNet-50 first growth timing · CIFAR-10 and CIFAR-100Min validation error rateNo clear effect
Adaptive cosine annealing learning-rate scheduler vs standard cosine and cosine with restarts (ablation)growing training schedule · CIFAR (appendix)AccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy