Research map / Architectures and efficiency / Pruning and efficient tuning
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use final-magnitude or magnitude-increase scores instead of random mask selection (ablation) | Iterative lottery-ticket pruning · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | Helped |
| Preserve each kept weight's original sign when reinitializing it (ablation) | Lottery-ticket subnetworks · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | Helped |
| Replace rewound magnitudes with same-sign constants for kept weights (ablation) | Lottery-ticket subnetworks · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | No clear effect |
| Freeze pruned weights at zero rather than at their initial values (ablation) | Iterative lottery-ticket pruning · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | Helped |
| Set pruned weights to zero when they moved toward zero, otherwise retain their initial values (ablation) | Iterative lottery-ticket pruning · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | Helped |
| Apply the toward-zero initialization rule to all weights, including kept weights (ablation) | Iterative lottery-ticket pruning · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | Helped |
| Set weights moving away from zero to zero instead of those moving toward zero (ablation) | Iterative lottery-ticket pruning · MNIST and CIFAR-10; FC, Conv2, Conv4, Conv6 | test accuracy | Hurt |
| Train binary masks while freezing the randomly initialized weights (ablation) | Untrained supermask networks · MNIST and CIFAR-10 | test accuracy: 79.3 -> 95.3; 22.3 -> 64.4; 23.7 -> 65.4; 24.0 -> 65.3 | Helped |
| Add dynamic weight rescaling to learned masks over initial weights (ablation) | Learned supermasks · MNIST and CIFAR-10 | test accuracy: 95.3 -> 97.8; 64.4 -> 65.0; 65.4 -> 71.7; 65.3 -> 76.3 | Helped |
| Add dynamic weight rescaling to learned masks over signed-constant weights (ablation) | Learned supermasks · MNIST and CIFAR-10 | test accuracy: 96.4 -> 98.0; 66.3 -> 66.0; 66.2 -> 72.5; 65.4 -> 76.5 | Mixed |
| Use signed-constant weights instead of initial weights with heuristic final masks (ablation) | Heuristic supermasks · MNIST and CIFAR-10 | test accuracy: 79.3 -> 86.3; 22.3 -> 37.4; 23.7 -> 39.7; 24.0 -> 41.0 | Helped |
Closest papers by meaning
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · similarity 0.92)
- Find A Winning Sign: Sign Is All We Need to Win the Lottery (ICLR 2025 · similarity 0.92)
- Optimal Lottery Tickets via Subset Sum: Logarithmic Over-Parameterization is Sufficient (NeurIPS 2020 · similarity 0.91)
- Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Win (ICML 2025 · similarity 0.91)
- The Elastic Lottery Ticket Hypothesis (NeurIPS 2021 · similarity 0.91)
- On the Sparsity of the Strong Lottery Ticket Hypothesis (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.