Research map / Architectures and efficiency / Pruning and efficient tuning
Pruning at Initialization – A Sketching Perspective
Noga Bar, Raja Giryes
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace threshold pruning with random mask sampled from SynFlow scores | SynFlow · Tiny-ImageNet ResNet-18 10% density | accuracy: 58.3 -> 60.64 | Improved |
| replace threshold pruning with random mask sampled from SynFlow scores | SynFlow · ImageNet ResNet-50 10% density | accuracy: 62.76 -> 63.05 | Improved |
| replace threshold pruning with random mask | SynFlow · CIFAR-100 ResNet-20 2% density | accuracy: 23.39 -> 24.89 | Improved |
| replace threshold pruning with random mask | SynFlow · CIFAR-10 VGG-19 5% density | accuracy: 92.68 -> 92.44 | No clear effect |
| replace the all-ones SynFlow input with chi-distributed random input | SynFlow · CIFAR-10 VGG-19 2% density | accuracy: 91.68 -> 91.32 | No clear effect |
| use sparse random data and random mask in data-free SNIP | SNIP + normal data · CIFAR-10 VGG-19 2% density | accuracy: 90.35 -> 92.03 | Improved |
| use sparse random data instead of normal random data in SNIP | SNIP · CIFAR-10 VGG-19 10% density | accuracy: 92.15 -> 92.85 | Improved |
| add random mask to ProsPr with random data | ProsPr + rand. data · CIFAR-100 VGG-19 10% density | accuracy: 11.55 ± 2.11 -> 72.68 ± 0.16 | Improved |
| add random mask to IMP with random data | IMP + rand. data · CIFAR-100 VGG-19 2% density | accuracy: 54.81 ± 6.87 -> 59.05 ± 0.14 | Improved |
| add random mask to GraSP with normal data | GraSP + normal data · CIFAR-10 ResNet-20 2% density | accuracy: 79.59 ± 0.1 -> 78.36 ± 0.16 | Hurt |
| replace Kaiming with Xavier initialization when adding random mask to SynFlow | SynFlow · CIFAR-10 VGG-19 10% density | accuracy: 91.5 ± 0.17 -> 93.33 ± 0.18 | Improved |
| add random mask to layerwise-sparsity SNIP | SNIP · CIFAR-10 VGG-19 2% density | accuracy: 64.64 ± 10.0 -> 87.97 ± 0.12 | Improved |
Closest papers by meaning
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR 2019 · similarity 0.92)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.91)
- Why Random Pruning Is All We Need to Start Sparse (ICML 2023 · similarity 0.91)
- Validating the Lottery Ticket Hypothesis with Inertial Manifold Theory (NeurIPS 2021 · similarity 0.91)
- Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry (ICML 2025 · similarity 0.89)
- Find A Winning Sign: Sign Is All We Need to Win the Lottery (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
