Research map / Architectures and efficiency / Pruning and efficient tuning
Network Pruning That Matters: A Case Study on Retraining Variants
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CLR instead of fine-tuning after PFEC pruning | PFEC-pruned networks · CIFAR-10 and CIFAR-100 | accuracy: 92.18±0.28 -> 92.71±0.33; 69.53±0.28 -> 70.06±0.35 | Helped |
| Use CLR instead of fine-tuning after PFEC pruning | PFEC-pruned ResNet-34 · ImageNet | top-1 accuracy: 72.96±0.06 -> 73.44±0.06 | Helped |
| Replace methodical pruning scores with random scores and retrain with CLR | Pruned ResNet-34 · ImageNet, PFEC comparison A | accuracy: 72.96±0.06 -> 73.47±0.08 | Helped |
| Replace methodical pruning scores with random scores and retrain with CLR | Pruned ResNet-50 · ImageNet, Taylor pruning 72% | accuracy: 74.50 -> 74.91 | Helped |
| Replace methodical pruning scores with random scores and retrain with CLR | Pruned ResNet-50 · ImageNet, Taylor pruning 91% | accuracy: 76.43 -> 75.93 | Hurt |
| Retrain Soft Filter Pruning models with CLR for 200 epochs rather than not retraining (ablation) | Soft Filter Pruning · CIFAR-10, ResNet-56, 30% filters pruned | accuracy: 92.74±0.28 -> 93.18±0.25 | Helped |
| Retrain Soft Filter Pruning models with fine-tuning for 200 epochs rather than not retraining (ablation) | Soft Filter Pruning · CIFAR-10, ResNet-56, 30% filters pruned | accuracy: 92.74±0.28 -> 92.85±0.21 | No clear effect |
| Use CLR rather than fine-tuning for Taylor pruning retraining (ablation) | Taylor-pruned ResNet-50 · ImageNet, Taylor-FO-BN-72% | accuracy: 74.50 -> 75.22 | Helped |
| Increase the CLR warmup duration across the tested settings (ablation) | PFEC-pruned networks · CIFAR-10 | accuracy | No clear effect |
| Use CLR rather than fine-tuning after PFEC pruning | PFEC-pruned ResNet-50 · ImageNet, Taylor-FO-BN-56% comparison | accuracy: 71.69 -> 70.70 | Hurt |
| Use CLR rather than fine-tuning after pruning | PFEC-pruned ResNet-34 · ImageNet, low retraining epochs | accuracy | Hurt |
Closest papers by meaning
- Comparing Rewinding and Fine-tuning in Neural Network Pruning (ICLR 2020 · similarity 0.93)
- How I Learned to Stop Worrying and Love Retraining (ICLR 2023 · similarity 0.92)
- Pruning from Scratch (AAAI 2020 · similarity 0.90)
- Rethinking the Value of Network Pruning (ICLR 2019 · similarity 0.90)
- A Three-regime Model of Network Pruning (ICML 2023 · similarity 0.89)
- Distilling the Knowledge in Data Pruning (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.