Research map / Architectures and efficiency / Pruning and efficient tuning
Picking Winning Tickets Before Training by Preserving Gradient Flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| preserve post-pruning gradient flow with GraSP rather than use SNIP | VGG19 and ResNet32 · CIFAR-10 and CIFAR-100 at 98% pruning | test accuracy: CIFAR-10: 92.05$\pm$0.28 -> 92.19$\pm$0.12 (VGG19); 87.51$\pm$0.31 -> 88.81$\pm$0.14 (ResNet32). CIFAR-100: 58.46$\pm$1. | Mixed |
| preserve post-pruning gradient flow with GraSP rather than use SNIP | ResNet-50 and VGG16 · ImageNet at 90% pruning | top-1 accuracy: 61.97 -> 68.14 (ResNet-50); 65.27 -> 69.94 (VGG16) | Helped |
| replace Kaiming initialization with Normal or Xavier initialization (ablation) | GraSP on ResNet32 · CIFAR-10, 60% and 90% pruning | test accuracy: 93.42 $\pm$ 0.39 -> 93.31 $\pm$ 0.36; 92.12 $\pm$ 0.39 -> 92.13 $\pm$ 0.36 (Normal); 93.42 $\pm$ 0.39 -> 93.32 $\pm$ 0.2 | No clear effect |
| vary the pruning-score batch size (ablation) | GraSP on ResNet32 · CIFAR-10 and CIFAR-100 | test accuracy | No clear effect |
Closest papers by meaning
- Pruning neural networks without any data by iteratively conserving synaptic flow (NeurIPS 2020 · similarity 0.90)
- Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enough (NeurIPS 2020 · similarity 0.90)
- Pruning from Scratch (AAAI 2020 · similarity 0.89)
- Winning the Lottery Ahead of Time: Efficient Early Network Pruning (ICML 2022 · similarity 0.89)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.88)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.