Research map / Architectures and efficiency / Pruning and efficient tuning
Rare Gems: Finding Lottery Tickets at Initialization
Kartik Sreenivasan, Jy-yong Sohn, Liu Yang, Matthew Grinde, Alliot Nagle, Hongyi Wang, Eric Xing, Kangwook Lee, Dimitris Papailiopoulos
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prune scores globally instead of layer-wise (ablation) | Edge-Popup · CIFAR-10 ResNet-20 0.59% sparsity | post-finetune acc (%): 24.47 -> 34.42 | Improved |
| add gradual pruning to global EP (ablation) | Global EP · CIFAR-10 ResNet-20 0.59% sparsity | post-finetune acc (%): 34.42 -> 63.54 | Improved |
| add L2 score regularization to global gradual EP (ablation) | Global EP with Gradual Pruning · CIFAR-10 ResNet-20 0.59% sparsity | post-finetune acc (%): 63.54 -> 63.72 | No clear effect |
| add L2 regularization to global gradual EP (ablation) | Global EP with Gradual Pruning · CIFAR-10 ResNet-20 0.59% sparsity | pre-finetune acc (%): 31.56 -> 19.67 | Hurt |
| run Gem-Miner 3000 epochs instead of 300 (prune every 150 epochs) (ablation) | Gem-Miner · CIFAR-10 ResNet-20 1.4% sparsity | post-finetune accuracy (%): 77.89 -> 79.50 | Improved |
| use Gem-Miner at initialization instead of ProsPr | pruning at initialization · CIFAR-10 ResNet-20 | post-finetune accuracy: 82.67% -> 83.4% | Improved |
| Gem-Miner instead of IMP with warmup | ResNet-20 · CIFAR-10 1.4% sparsity | post-finetune accuracy (%): 74.52 -> 77.89 | Improved |
| Gem-Miner in dense regime (50% or 20% sparsity) | Gem-Miner · CIFAR-10 | post-finetune accuracy | Hurt |
| Gem-Miner on Caltech-101 finetuning vs IMP | ResNet-50 · Caltech-101 | post-finetune accuracy | Hurt |
Closest papers by meaning
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · similarity 0.92)
- Plant 'n' Seek: Can You Find the Winning Ticket? (ICLR 2022 · similarity 0.89)
- Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network (ICLR 2021 · similarity 0.89)
- Winning the Lottery with Continuous Sparsification (NeurIPS 2020 · similarity 0.89)
- On the Sparsity of the Strong Lottery Ticket Hypothesis (NeurIPS 2024 · similarity 0.88)
- Find A Winning Sign: Sign Is All We Need to Win the Lottery (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
