Research map / Architectures and efficiency / Pruning and efficient tuning
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot
Jingtong Su, Yihang Chen, Tianle Cai, Tianhao Wu, Ruiqi Gao, Liwei Wang, Jason Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| randomly rearrange kept connections per layer, keeping per-layer counts (random tickets from smart ratios) (ablation) | initial tickets (SNIP/GraSP/LT) · CIFAR-10 / CIFAR-100 | test accuracy | No clear effect |
| corrupt pruning data with random labels and random pixels (ablation) | initial tickets (SNIP/GraSP/LT) · CIFAR-10 | test accuracy | No clear effect |
| use data-independent random tickets with smart ratios instead of GraSP | VGG19 · CIFAR-10 98% pruned | test accuracy: 91.90 -> 92.45 | Improved |
| use random tickets with smart ratios instead of SNIP | ResNet32 · Tiny-ImageNet 98% pruned | test accuracy: 32.09 -> 43.93 | Improved |
| use random tickets instead of GraSP | ResNet32 · Tiny-ImageNet 98% pruned | test accuracy: 39.43 -> 43.93 | Improved |
| use random tickets instead of GraSP | ResNet32 · CIFAR-10 98% pruned | test accuracy: 89.21 -> 89.10 | No clear effect |
| replace descending smart ratio with balanced keep-ratio (ablation) | random tickets · CIFAR-10 ResNet32 98% | test accuracy: 89.10 -> 86.52 | Hurt |
| replace descending smart ratio with ascending keep-ratio (ablation) | random tickets · CIFAR-10 ResNet32 | test accuracy: 89.10 -> 10.00 | Hurt |
| use cubic or linear decay instead of smart ratio (ablation) | random tickets · CIFAR-10 | test accuracy: 89.10 -> 89.14 | No clear effect |
| shuffle unmasked weights per layer (ablation) | learning rate rewinding · CIFAR-10 ResNet32 95% | test accuracy: 93.02 -> 91.29 | Hurt |
| prune with half the dataset (ablation) | learning rate rewinding · CIFAR-10 ResNet32 95% | test accuracy: 93.02 -> 92.03 | Hurt |
| apply smart-ratio layerwise magnitude pruning (hybrid tickets) instead of global | learning rate rewinding · CIFAR-100 ResNet32 98% | test accuracy: 59.22 -> 63.44 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- SNIP — SNIP: SINGLE-SHOT NETWORK PRUNING BASED ON CONNECTION SENSITIVITY (ICLR 2019) · better on 5 of 5 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Rethinking the Value of Network Pruning (ICLR 2019 · similarity 0.85)
- Network Pruning That Matters: A Case Study on Retraining Variants (ICLR 2021 · similarity 0.84)
- Pruning from Scratch (AAAI 2020 · similarity 0.83)
- A Gradient Flow Framework For Analyzing Network Pruning (ICLR 2021 · similarity 0.83)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.82)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
