Research map / Architectures and efficiency / Pruning and efficient tuning
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
Xiaolong Ma, Geng Yuan, Xuan Shen, Tianlong Chen, Xuxi Chen, Xiaohan Chen, Ning Liu, Minghai Qin, Sijia Liu, Zhangyang Wang, Yanzhi Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train dense network with sufficient epochs (160 vs ~78) | ResNet-20 lr 0.005 · CIFAR-10 | accuracy (%): 88.0 -> 89.6 | Improved |
| use larger learning rate 0.1 for dense pretraining | ResNet-20 · CIFAR-10 | accuracy (%): 88.0 -> 92.3 | Improved |
| use original initialization (LT) instead of random reinitialization (RR) for IMP subnetwork | ResNet-20 lr 0.005 · CIFAR-10 | accuracy (%): 82.9 -> 87.4 | Improved |
| use original initialization instead of reinitialization at large lr 0.1 | ResNet-20 · CIFAR-10 | accuracy (%) | No clear effect |
| use IMP instead of OMP | networks with residual connections · CIFAR-10 small dataset | accuracy | Improved |
| use IMP instead of OMP (ablation) | ResNet-32-like without residual connections · CIFAR-10 | accuracy | No clear effect |
| use IMP instead of OMP | ResNet-18/50 · Tiny-ImageNet and ImageNet-1K | accuracy | Hurt |
| use different lr for subnetwork training than pretraining lr (0.005 vs 0.01) (ablation) | LT-IMP ResNet-20 s=0.914 · CIFAR-10 | accuracy (%): 89.4 -> 89.7 | Improved |
| use different lr for subnetwork than pretraining lr (0.05 vs 0.1) (ablation) | LT-OMP ResNet-20 s=0.914 · CIFAR-10 | accuracy (%): 87.2 -> 87.4 | Improved |
| rewind subnetwork to early-epoch weights (about 5% of training) (ablation) | subnetworks of all networks · CIFAR/Tiny-ImageNet/ImageNet | accuracy | Improved |
| increase network capacity (ablation) | ResNet-20 vs ResNet-32/18/VGG-16 · CIFAR-10 | maximum sparsity with secondary prize ticket: 0.914 -> 0.945 or higher | Improved |
Closest papers by meaning
- Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP (ICLR 2020 · similarity 0.90)
- Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks (NeurIPS 2021 · similarity 0.89)
- Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not? (ICML 2021 · similarity 0.89)
- You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership (NeurIPS 2021 · similarity 0.89)
- Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Win (ICML 2025 · similarity 0.88)
- Find A Winning Sign: Sign Is All We Need to Win the Lottery (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
