Research map / Architectures and efficiency / Pruning and efficient tuning
Importance Estimation for Neural Network Pruning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use squared Taylor first-order filter importance instead of weight magnitude | ResNet20 pruning · CIFAR-10, prune B (119 neurons) | accuracy (%): 62.03($\pm$1.36) -> 89.78 ($\pm$0.18) | Helped |
| use first-order Taylor importance rather than second-order Taylor importance (ablation) | ResNet20 pruning · CIFAR-10, prune B (119 neurons) | accuracy (%): 89.76 ($\pm$0.17) -> 89.78 ($\pm$0.18) | No clear effect |
| put the gate after BN rather than before BN2 (ablation) | Taylor FO pruning criterion · CIFAR-10, ResNet-18 | Spearman correlation with oracle: 0.275 -> 0.955 | Helped |
| use full gradients instead of minibatch-averaged gradients (ablation) | Taylor FO gate criterion · CIFAR-10, ResNet-18 | Spearman correlation with oracle: 0.955 -> 0.775 | Hurt |
| recompute importance criteria after each pruning iteration rather than keep them fixed (ablation) | ResNet-18 pruning · CIFAR-10 | pruning performance | Helped |
| prune skip connections together with the other channels (ablation) | Taylor-FO-BN pruning · ImageNet, ResNet-50 | top-1 error (%): 30.74 -> 28.31 | Helped |
| iteratively remove filters and reevaluate the criterion instead of pruning 10000 neurons in one step (ablation) | Taylor-FO-BN pruning · ImageNet, ResNet-101 | top-1 error (%): 25.3% ... $0.68$\% higher than Taylor-FO-BN-50% | Helped |
| prune iteratively rather than continuously according to the loss threshold (ablation) | Taylor-FO-BN pruning · ImageNet, ResNet-101 | pruning performance | Helped |
| prune filters using Taylor-FO-BN-50% rather than BN-ISTA v2 | ResNet-101 · ImageNet | top-1 error (%): 25.44 -> 24.62 | Helped |
| use Taylor-FO-BN-40% rather than the unpruned network | ResNet-101 · ImageNet | GFLOPs: 7.80 -> 1.76 | Helped |
| use Taylor-FO-BN-50% rather than weight-magnitude pruning | ResNet-101 · ImageNet oracle-correlation study | Spearman correlation: 0.376 -> 0.925 | Helped |
Closest papers by meaning
- NISP: Pruning Networks Using Neuron Importance Score Propagation (CVPR 2018 · similarity 0.92)
- Pruning from Scratch (AAAI 2020 · similarity 0.91)
- Provable Filter Pruning for Efficient Neural Networks (ICLR 2020 · similarity 0.91)
- A Probabilistic Approach to Neural Network Pruning (ICML 2021 · similarity 0.90)
- When To Prune? A Policy Towards Early Structural Pruning (CVPR 2022 · similarity 0.90)
- Towards Meta-Pruning via Optimal Transport (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.