AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

DεpS: Delayed ε-Shrinking for Faster Once-For-All Training

Aditya Annavajjala, Alind Khare, Animesh Agrawal, Igor Fedorov, Hugo M Latapie, Myungjin Lee, Alexey Tumanov

ECCV 2024 · Architectures and efficiency › RNNs and normalization · approach: Efficient network compression and training · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
start shrinking at 50% of full-model training instead of 25% or 75% (ablation)DepS · ImageNet-1ksubnet accuracy: about 1% degradation around 200 MFLOPs at 75% vs 50%Improved
add epsilon-shrinking (incremental subnet learning-rate warmup) (ablation)DepS · ImageNet-1k full modelfull model top-1 accuracy: avoids a 2% accuracy drop at epoch 150Improved
use IKD-Warmup instead of plain inplace KD (ablation)DepS · ImageNet-1ksubnet top-1 accuracy: +3.5% at 560 MFLOPs, +2% at 150 MFLOPsImproved
delay shrinking (train full model partially) versus complete full-model trainingMobileNet-based supernet initialization · CIFAR-10, CIFAR-100, ImageNet-1ksmall subnet accuracyImproved
replace OFA with DepS on ImageNet-1k MobileNetV3once-for-all training · ImageNet-1ktraining cost (GPU hours): 1256 -> 700Improved
apply DepS on ProxylessNAS space instead of OFAonce-for-all training · ImageNet-1k (ProxylessNAS space)smallest subnet accuracy: +1.8%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy