Research map / Architectures and efficiency / Pruning and efficient tuning
Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsity
Artem Vysogorets, Julia Kempe
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| evaluate with effective instead of direct compression for pruning methods | SynFlow vs SNIP and GraSP · LeNet-300-100 / MNIST | test accuracy vs compression | Mixed |
| evaluate with effective instead of direct compression on convolutional nets | SynFlow vs other methods · VGG-19 and ResNet-18 | performance gap | Mixed |
| use LAMP pruning | LAMP · all architectures | effective vs direct compression | Hurt |
| use IGQ layerwise sparsity for random pruning instead of Uniform/Uniform+ (ablation) | random pruning · ResNet-50 up to 99% sparsity | test accuracy | Improved |
| use ERK instead of IGQ for random pruning (ablation) | random pruning · VGG-19 at 1000x or higher compression | test accuracy | Improved |
| use IGQ layerwise quotas in magnitude pruning instead of Uniform/ERK baselines (ablation) | magnitude pruning with retraining · all architectures, up to 100x | test accuracy | Improved |
| use global magnitude pruning instead of IGQ (ablation) | magnitude pruning · VGG-19 | test accuracy | Hurt |
| use effective pruning (binary search to target effective sparsity) (ablation) | ranking-based and random pruning · ResNet-18 | reaching target effective sparsity | Improved |
Closest papers by meaning
- The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training (ICLR 2022 · similarity 0.94)
- Pruning via Sparsity-indexed ODE: a Continuous Sparsity Viewpoint (ICML 2023 · similarity 0.93)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.93)
- Pruning from Scratch (AAAI 2020 · similarity 0.92)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.91)
- Towards Meta-Pruning via Optimal Transport (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
