Research map / Architectures and efficiency / Pruning and efficient tuning
How I Learned to Stop Worrying and Love Retraining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ALLR instead of fixed or conventional pruning-stable training schedules | IMP retraining on ResNet-50 · ImageNet, 90% sparsity, 10 epochs retraining | accuracy: 71.51 -> 71.99 | Helped |
| Use linear or cosine-based schedules rather than constant or stepped schedules | IMP retraining · ImageNet and other evaluated tasks | accuracy | Helped |
| Adapt the initial linear learning rate to pruning impact and retraining time | LLR retraining · ImageNet, short retraining budgets and medium sparsity | accuracy | Helped |
| Replace long iterative retraining with an ALLR budget of around 100 retraining epochs | IMP · CIFAR-10, 90%, 95%, and 98% sparsity | accuracy: around 100; 2000, 2800, and 3600 | No clear effect |
| Use Budgeted IMP with ALLR instead of pruning-stable approaches | Training from random initialization · CIFAR-10, 90% sparsity | accuracy: 92.84 -> 93.35 | Helped |
| Use Budgeted IMP with ALLR instead of pruning-stable approaches | Training from random initialization · ImageNet, 70% sparsity | accuracy: 75.60 -> 75.62 | No clear effect |
| Use a linearly decaying rather than a stepped learning-rate schedule for budgeted IMP retraining | IMP retraining · CIFAR-10 and ImageNet | accuracy | Helped |
Closest papers by meaning
- Network Pruning That Matters: A Case Study on Retraining Variants (ICLR 2021 · similarity 0.92)
- Sparse Training via Boosting Pruning Plasticity with Neuroregeneration (NeurIPS 2021 · similarity 0.88)
- Pruning Neural Networks at Initialization: Why Are We Missing the Mark? (ICLR 2021 · similarity 0.87)
- Comparing Rewinding and Fine-tuning in Neural Network Pruning (ICLR 2020 · similarity 0.87)
- When to retrain a machine learning model (ICML 2025 · similarity 0.85)
- Distilling the Knowledge in Data Pruning (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.