Research map / Architectures and efficiency / Pruning and efficient tuning
When To Prune? A Policy Towards Early Structural Pruning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EPI to choose the pruning start epoch instead of a pre-defined start | Magnitude-based structural pruning · ImageNet, networks and pruning ratios averaged | Average accuracy drop relative to oracle: 0.204 -> 0.132 | Helped |
| Use EPI to choose the pruning start epoch instead of a pre-defined start | Gradient-based structural pruning · ImageNet, networks and pruning ratios averaged | Average accuracy drop relative to oracle: 0.214 -> 0.142 | Helped |
| Use EPI-guided pruning rather than pruning a pretrained model | ResNet50 · ImageNet, 10% pruning, equal epochs | Top-1 accuracy: 77.17% -> 77.69% | Helped |
| Use EPI-guided pruning rather than pruning a pretrained model | ResNet50 · ImageNet, 20% pruning, equal epochs | Top-1 accuracy: 76.78% -> 76.82% | Helped |
| Use EPI-guided pruning rather than pruning a pretrained model | ResNet50 · ImageNet, 30% pruning, equal epochs | Top-1 accuracy: 76.18% -> 76.23% | Helped |
| Train the pruned structure during training rather than training its initialization-derived ticket in isolation | ResNet50 · ImageNet, structural pruning | Top-1 accuracy | Helped |
| Keep the number of neurons per layer fixed while varying neuron identities (ablation) | ResNet50, gradient-based pruning at 50% · ImageNet | Top-1 accuracy difference: 0.36% ± 0.13% | No clear effect |
| Vary the number of neurons per layer to produce structures with around 0.8 similarity (ablation) | ResNet50 · ImageNet, gradient-based pruning | Top-1 accuracy difference: 0.53% ± 0.39% | Mixed |
| Prune 50% of neurons at initialization rather than after early dense training (ablation) | ResNet50 with magnitude-based ranking · ImageNet | Top-1 accuracy | Hurt |
| Start pruning later rather than at initialization (ablation) | ResNet50 with gradient-based ranking · ImageNet, high pruning ratios | Top-1 accuracy | Helped |
Closest papers by meaning
- Rethinking the Value of Network Pruning (ICLR 2019 · similarity 0.92)
- Pruning from Scratch (AAAI 2020 · similarity 0.91)
- Sparse Training via Boosting Pruning Plasticity with Neuroregeneration (NeurIPS 2021 · similarity 0.90)
- On the Predictability of Pruning Across Scales (ICML 2021 · similarity 0.90)
- Pruning neural networks without any data by iteratively conserving synaptic flow (NeurIPS 2020 · similarity 0.90)
- Pruning Neural Networks at Initialization: Why Are We Missing the Mark? (ICLR 2021 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.