Research map / Architectures and efficiency / Pruning and efficient tuning
Rigging the Lottery: Making All Tickets Winners
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RigL instead of magnitude pruning | ResNet-50 · ImageNet-2012, 95% sparsity | Top-1 accuracy: 72.7 -> 73.1 | Helped |
| Use RigL instead of magnitude pruning | ResNet-50 · ImageNet-2012, 96.5% sparsity | Top-1 accuracy: 69.26 -> 71.1 | Helped |
| Use ERK sparsity distribution rather than uniform sparsity (ablation) | RigL, ResNet-50 · ImageNet-2012 | Top-1 accuracy: 73.1 -> 74.5; training FLOPs 1.14x -> 2.09x | Mixed |
| Train for 5× the steps (ablation) | RigL, ResNet-50 · ImageNet-2012, 95% sparsity | Top-1 accuracy: 67.5+-0.10 -> 73.1+-0.12 | Helped |
| Train for 5× the steps (ablation) | RigL, ResNet-50 · ImageNet-2012, 96.5% sparsity | Top-1 accuracy: 65.0+-0.28 -> 71.1+-0.20 | Helped |
| Use lottery-ticket initialization instead of random initialization (ablation) | RigL, ResNet-50 · ImageNet-2012 | Test accuracy: 74.55\ci{0.06} -> 73.93\ci{0.09} | Hurt |
| Keep the sparse topology fixed rather than dynamically rewiring it (ablation) | Lottery-initialized ResNet-50 · ImageNet-2012 | Test accuracy: 73.93\ci{0.09} -> 70.82\ci{0.07} | Hurt |
| Train for twice as long rather than restarting from lottery-ticket initialization (ablation) | RigL, ResNet-50 · ImageNet-2012 | Test accuracy: 74.55\ci{0.06} -> 76.06\ci{0.09} | Helped |
| Use ERK rather than other sparsity distributions (ablation) | RigL · ResNet-50 / ImageNet-2012 | Test accuracy | Helped |
| Update the mask every 100 iterations rather than at other tested intervals (ablation) | RigL · ResNet-50 / ImageNet-2012 | Test accuracy | Helped |
| Use a cosine update schedule rather than the other tested annealing schedules (ablation) | RigL · ImageNet-2012 | Test accuracy | Mixed |
| Use instantaneous gradients rather than accumulated momentum to grow connections (ablation) | Sparse training · ImageNet-2012 | Test accuracy | No clear effect |
Closest papers by meaning
- Winning the Lottery with Continuous Sparsification (NeurIPS 2020 · similarity 0.90)
- Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win (AAAI 2022 · similarity 0.90)
- Calibrating the Rigged Lottery: Making All Tickets Reliable (ICLR 2023 · similarity 0.90)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.89)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.88)
- Sparse maximal update parameterization: A holistic approach to sparse training dynamics (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.