AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?

Xiaolong Ma, Geng Yuan, Xuan Shen, Tianlong Chen, Xuxi Chen, Xiaohan Chen, Ning Liu, Minghai Qin, Sijia Liu, Zhangyang Wang, Yanzhi Wang

NeurIPS 2021 · Architectures and efficiency › Pruning and efficient tuning · approach: Lottery ticket discovery · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train dense network with sufficient epochs (160 vs ~78)ResNet-20 lr 0.005 · CIFAR-10accuracy (%): 88.0 -> 89.6Improved
use larger learning rate 0.1 for dense pretrainingResNet-20 · CIFAR-10accuracy (%): 88.0 -> 92.3Improved
use original initialization (LT) instead of random reinitialization (RR) for IMP subnetworkResNet-20 lr 0.005 · CIFAR-10accuracy (%): 82.9 -> 87.4Improved
use original initialization instead of reinitialization at large lr 0.1ResNet-20 · CIFAR-10accuracy (%)No clear effect
use IMP instead of OMPnetworks with residual connections · CIFAR-10 small datasetaccuracyImproved
use IMP instead of OMP (ablation)ResNet-32-like without residual connections · CIFAR-10accuracyNo clear effect
use IMP instead of OMPResNet-18/50 · Tiny-ImageNet and ImageNet-1KaccuracyHurt
use different lr for subnetwork training than pretraining lr (0.005 vs 0.01) (ablation)LT-IMP ResNet-20 s=0.914 · CIFAR-10accuracy (%): 89.4 -> 89.7Improved
use different lr for subnetwork than pretraining lr (0.05 vs 0.1) (ablation)LT-OMP ResNet-20 s=0.914 · CIFAR-10accuracy (%): 87.2 -> 87.4Improved
rewind subnetwork to early-epoch weights (about 5% of training) (ablation)subnetworks of all networks · CIFAR/Tiny-ImageNet/ImageNetaccuracyImproved
increase network capacity (ablation)ResNet-20 vs ResNet-32/18/VGG-16 · CIFAR-10maximum sparsity with secondary prize ticket: 0.914 -> 0.945 or higherImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy