Research map / Architectures and efficiency / Pruning and efficient tuning
Robust Pruning at Initialization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Stable ResNet parameterization with SBP instead of SNIP | ResNet104 · CIFAR10, 90% sparsity | test accuracy: $93.25 \pm 0.53$ -> 94.69 $\pm$ 0.13 | Helped |
| Use Stable ResNet parameterization with SBP instead of GraSP | ResNet104 · CIFAR10, 98% sparsity | test accuracy: 91.19$\pm$0.35 -> 92.08 $\pm$ 0.14 | Helped |
| Use Stable ResNet parameterization with SBP instead of GraSP | ResNet104 · CIFAR100, 90% sparsity | test accuracy: 73.33$\pm$0.21 -> 74.17 $\pm$ 0.11 | Helped |
| Use Stable ResNet parameterization with SBP instead of GraSP | ResNet104 · CIFAR100, 95% sparsity | test accuracy: $ 70.95 \pm 1.12 $ -> 71.84 $\pm$ 0.13 | Helped |
| Use Stable ResNet parameterization with SBP instead of SynFlow | ResNet104 · CIFAR100, 98% sparsity | test accuracy: 67.02$\pm $0.10 -> 67.73 $\pm$ 0.28 | Helped |
| Add the rescaling trick after EOC initialization and pruning (ablation) | V-CNN · MNIST, 50% sparsity, depth 100 | test accuracy: 10.00\pm$0.0 -> 99.15\pm$0.08 | Helped |
| Use EOC initialization instead of Ordered phase initialization (ablation) | V-CNN · MNIST, 50% sparsity, depth 100 | test accuracy: 10.00\pm$0.0 -> 10.00\pm$0.0 | No clear effect |
| Increase VGG16 convolutional depth within pooling blocks to form 3xVGG16 (ablation) | EOC + ReScaling pruning · CIFAR10, 85% sparsity | test accuracy: 93.15$\pm$0.12 -> 93.97$\pm$0.17 | Helped |
| Apply EOC initialization and rescaling instead of SynFlow | VGG16 · CIFAR10, 85% sparsity | test accuracy: 93.21$\pm$0.13 -> 93.15$\pm$0.12 | No clear effect |
| Apply GraSP to Stable ResNet rather than standard ResNet (ablation) | GraSP · CIFAR10 and CIFAR100, ResNet32/50 | test accuracy | No clear effect |
Closest papers by meaning
- Pruning Neural Networks at Initialization: Why Are We Missing the Mark? (ICLR 2021 · similarity 0.92)
- A Signal Propagation Perspective for Pruning Neural Networks at Initialization (ICLR 2020 · similarity 0.91)
- Pruning from Scratch (AAAI 2020 · similarity 0.90)
- A Probabilistic Approach to Neural Network Pruning (ICML 2021 · similarity 0.90)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.89)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.