Research map / Architectures and efficiency / Pruning and efficient tuning
Pruning from Scratch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn the pruned structure from random initialization instead of pre-trained weights | Channel pruning · CIFAR10, ResNet20, ResNet56, ResNet110, VGG16, and VGG19 | accuracy: 91.14±0.32 vs 91.07±0.23; 93.05±0.19 vs 93.49 | Mixed |
| Prune from scratch rather than use Lottery Ticket initialization | CIFAR10 structured pruning · CIFAR10, six network/pruning-ratio settings | accuracy: 91.14±0.32 vs 90.94±0.26 | Helped |
| Use 0.75× channel shrinkage instead of a larger width expansion (ablation) | Pruning from scratch · CIFAR10 and ImageNet models at 50% FLOPS reduction | accuracy | Mixed |
| Increase the width expansion excessively (ablation) | Pruning from scratch · CIFAR10 and ImageNet models at 50% FLOPS reduction | accuracy | Hurt |
| Vary the pruning ratio (ablation) | Pruning from scratch · CIFAR10 models | accuracy | No clear effect |
| Use a small channel-gate sparsity level (ablation) | Pruning from scratch · CIFAR10 models at 50% FLOPS reduction | accuracy | Hurt |
| Vary the channel-gate sparsity ratio (ablation) | Pruning from scratch · CIFAR10 models at 50% FLOPS reduction | accuracy | No clear effect |
| Fix random weights during channel-importance optimization instead of updating them | Pruning from scratch · CIFAR10 ResNet56 and ImageNet ResNet50 structure search | search time: 2.3 hours and 1.0 hours vs 0.12 hours; 310 hours vs 2.8 hours | Helped |
Closest papers by meaning
- Rethinking the Value of Network Pruning (ICLR 2019 · similarity 0.94)
- A Probabilistic Approach to Neural Network Pruning (ICML 2021 · similarity 0.93)
- Network Pruning via Transformable Architecture Search (NeurIPS 2019 · similarity 0.92)
- “Learning-Compression” Algorithms for Neural Net Pruning (CVPR 2018 · similarity 0.92)
- What Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity (ICLR 2024 · similarity 0.92)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.