Research map / Architectures and efficiency / Pruning and efficient tuning
A Unified Framework for Soft Threshold Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt threshold increments in proportion to the learning rate to keep the L1 penalty constant | Soft threshold pruning · ImageNet, ResNet-50 | Top-1 accuracy | Helped |
| Replace exact LATS summation with its integral approximation (S-LATS) | STDS · ImageNet, ResNet-50 at final thresholds 0.1, 0.5, 1.0, and 5.0 | Top-1 accuracy: 76.53 -> 76.75; 73.12 -> 73.03; 69.56 -> 69.64; 53.80 -> 53.64 | No clear effect |
| Use batch size 1024 instead of the standard batch size 256 | S-LATS · ImageNet, ResNet-50 | Top-1 accuracy | Helped |
| Use PGH continuation schedules with smaller beta to reach sparsity earlier | S-LATS · ImageNet, ResNet-50 | convergence speed to a sparse network | Helped |
| Prune at initialization by taking beta toward zero | PGH · ImageNet, ResNet-50 | Top-1 accuracy: 74.69 at 87.16% sparsity; 72.89 at 90.00%; 68.23 at 93.11%; 62.22 at 95.64% | Mixed |
| Increase final threshold to create greater sparsity | PGH pruning at initialization · ImageNet, ResNet-50 | number of zeroed layers: 0 at D=0.1; 9 at D=0.11; 9 at D=0.13; 27 at D=0.15 | Mixed |
| Replace the trainable threshold mechanism with a sine schedule (ablation) | STR · ImageNet, ResNet-50 | Top-1 accuracy: STR: 71.51 (Sine), 69.85 (Linear), 68.55 (Log2) | Helped |
| Replace the default STDS threshold schedule with a sine schedule (ablation) | STDS · ImageNet, ResNet-50 | Top-1 accuracy: 72.42 (Sine), 59.79 (Linear), 64.74 (Log2) | Helped |
| Replace the sine schedule with a linear schedule (ablation) | STDS · ImageNet, ResNet-50 | Top-1 accuracy: 72.42 -> 59.79 | Hurt |
| Replace the sine schedule with a Log2 schedule (ablation) | STDS · ImageNet, ResNet-50 | Top-1 accuracy: 72.42 -> 64.74 | Hurt |
| Remove weight decay when comparing threshold schedulers (ablation) | ResNet-50 threshold-scheduler ablation · ImageNet, ResNet-50 | Top-1 accuracy | No clear effect |
| Replace the original STDS sine scheduler with S-LATS (ablation) | STDS · ImageNet, ResNet-50 | Top-1 accuracy | Helped |
Closest papers by meaning
- The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training (ICLR 2022 · similarity 0.90)
- Pruning via Sparsity-indexed ODE: a Continuous Sparsity Viewpoint (ICML 2023 · similarity 0.89)
- SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization (NeurIPS 2024 · similarity 0.88)
- SAFE: Finding Sparse and Flat Minima to Improve Pruning (ICML 2025 · similarity 0.88)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.87)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.