Research map / Architectures and efficiency / Pruning and efficient tuning
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use scheduled cyclic grow-and-prune instead of RigL 12× | ResNet-50 · ImageNet, uniform 80% sparsity | Top-1 accuracy (%): 77.1 -> 77.9 | Helped |
| use scheduled cyclic grow-and-prune instead of RigL 12× | ResNet-50 · ImageNet, non-uniform 90% sparsity | Top-1 accuracy (%): 76.8 -> 77.9 | Helped |
| use C-GaP instead of prune-from-dense | SSD · COCO-2017, 90% sparsity | mAP@[.5:.95]: 24.3 -> 25.9 | Helped |
| use C-GaP instead of prune-from-dense | PointNet++ · ShapeNet, 80% sparsity | Class mIoU (%): 79.1 -> 82.9 | Helped |
| use C-GaP instead of prune-from-dense | Transformer · WMT-14 En-De, 90% sparsity, 3 partitions | SacreBLEU: 25.7 -> 27.7 | Helped |
| increase the C-GaP partition count from 1 to 4 (ablation) | ResNet-50 · ImageNet, 90% uniform sparsity | Top-1 accuracy (%): 75.9 -> 76.3 | Helped |
| increase the C-GaP partition count from 1 to 3 (ablation) | Transformer · WMT-14 En-De, 90% uniform sparsity | SacreBLEU: 26.8 -> 27.7 | Helped |
| increase the C-GaP partition count from 3 to 6 (ablation) | Transformer · WMT-14 En-De, 90% uniform sparsity | SacreBLEU: 27.7 -> 27.1 | Hurt |
| use cyclic contiguous partitions instead of random partitions (ablation) | Transformer C-GaP · WMT-14 En-De, 90% uniform sparsity, 3 partitions | SacreBLEU: 27.0 -> 27.7 | Helped |
| use cyclic contiguous partitions instead of random partitions (ablation) | ResNet-50 C-GaP · ImageNet, 90% non-uniform sparsity | Top-1 accuracy (%): 77.8 -> 77.9 | Helped |
| use C-GaP instead of prune-from-dense for block sparsity | Transformer · WMT-14 En-De, 80% block 1×8 sparsity | BLEU: 26.1 -> 27.2 | Helped |
| prune the fully connected layer instead of keeping it dense (ablation) | ResNet-50 C-GaP · ImageNet, uniform 80% sparsity | Top-1 accuracy (%): 77.9 -> 77.8 | No clear effect |
Closest papers by meaning
- Growing Efficient Deep Networks by Structured Continuous Sparsification (ICLR 2021 · similarity 0.90)
- Computation and Memory-Efficient Model Compression with Gradient Reweighting (NeurIPS 2025 · similarity 0.90)
- Neural Pruning via Growing Regularization (ICLR 2021 · similarity 0.90)
- The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training (ICLR 2022 · similarity 0.90)
- Dynamic Model Pruning with Feedback (ICLR 2020 · similarity 0.89)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.