Research map / Architectures and efficiency / Pruning and efficient tuning
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Dynamically choose per-level pruning ratio so the pruned solution stays within the train LCS-set (adaptive heuristic) instead of fixed 20% | IMP on ResNet-18 · CIFAR-100 | number of pruning levels to reach same sparsity: 7 levels vs 11 levels | Helped |
| Adaptive pruning ratio heuristic vs fixed 20% per level | IMP on ResNet-18 · CIFAR-100 | training compute: approximately 33% reduction | Helped |
| Pruning in flatter Hessian directions (magnitude pruning vs random pruning) | IMP magnitude pruning · CIFAR-10/CIFAR-100 error landscape | allowable pruning ratio per iteration | Helped |
| Sharper Hessian eigenspectrum at higher sparsity levels | IMP maximal pruning ratio · ResNet pruning theory | maximal allowed pruning ratio | Hurt |
| Retraining with finetuning (FT) instead of weight rewinding or LR rewinding | IMP retraining step · CIFAR-10 / ImageNet | matching subnetwork found / weight distribution reequilibration | Hurt |
| Learning rate rewinding instead of finetuning for retraining | IMP retraining step · CIFAR-10 / ImageNet | test error across sparsities | Helped |
| Random perturbation of rewind point with norm equal to pruning projection (SGD robustness) | IMP rewind point at stable steps · CIFAR-10 ResNet-20 / ImageNet | linear mode connectivity of retrained network | No clear effect |
Closest papers by meaning
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · similarity 0.93)
- Plant 'n' Seek: Can You Find the Winning Ticket? (ICLR 2022 · similarity 0.92)
- Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets (ICML 2022 · similarity 0.92)
- Find A Winning Sign: Sign Is All We Need to Win the Lottery (ICLR 2025 · similarity 0.91)
- Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks (AAAI 2025 · similarity 0.90)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.