Research map / Architectures and efficiency / Pruning and efficient tuning
Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use meta-gradients through 3 optimization steps instead of 0 (ablation) | ProsPr structured pruning · CIFAR-10, VGG-19, 90% sparsity | accuracy: 93.1\% $\pm$ 0.04 -> 93.6\% $\pm$ 0.19 | Helped |
| Use meta-gradients through 3 optimization steps instead of 0 (ablation) | ProsPr structured pruning · CIFAR-100, VGG-19, 90% sparsity | accuracy: 68.3\% $\pm$ 0.12 -> 71.0\% $\pm$ 0.21 | Helped |
| Use meta-gradients through 3 optimization steps instead of 0 (ablation) | ProsPr structured pruning · CIFAR-10, VGG-19, 95% sparsity | accuracy: 92.5 $\pm$ 0.12 -> 93.29 $\pm$ 0.29 | Helped |
| Use meta-gradients through 3 optimization steps instead of 0 (ablation) | ProsPr structured pruning · CIFAR-100, VGG-19, 95% sparsity | accuracy: 63.20\% $\pm$ 0.52 -> 67.98\% $\pm$ 0.31 | Helped |
| Use ProsPr instead of 3SP for structured pruning | VGG-19 · CIFAR-10, 95% sparsity | accuracy: 92.5 $\pm$ 0.12 -> 93.32 $\pm$ 0.15 | Helped |
| Use ProsPr instead of 3SP for structured pruning | VGG-19 · CIFAR-100, 95% sparsity | accuracy: 63.2 $\pm$ 0.52 -> 68.03 $\pm$ 0.38 | Helped |
| Use ProsPr instead of FORCE | ResNet-50 · ImageNet, 90% sparsity | Top-1 accuracy: 64.9 -> 66.86 | Helped |
| Use ProsPr instead of FORCE | ResNet-50 · ImageNet, 95% sparsity | Top-1 accuracy: 59.0 -> 59.62 | Helped |
| Use ProsPr instead of GraSP-MB | VGG-19 · ImageNet, 95% sparsity | Top-1 accuracy: 67.6 -> 66.1 | Hurt |
| Initialize ProsPr from BYOL-trained weights instead of random weights (ablation) | ResNet-18 · CIFAR-10, 95% pruning | test accuracy: 93.6 -> 93.62 | No clear effect |
| Initialize ProsPr from BYOL-trained weights instead of random weights (ablation) | ResNet-18 · CIFAR-100, 95% pruning | test accuracy: 73.2 -> 74.02 | Helped |
Closest papers by meaning
- The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training (ICLR 2022 · similarity 0.92)
- Pruning Neural Networks at Initialization: Why Are We Missing the Mark? (ICLR 2021 · similarity 0.90)
- Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining (ICLR 2022 · similarity 0.90)
- Pruning from Scratch (AAAI 2020 · similarity 0.90)
- Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures (CVPR 2023 · similarity 0.90)
- OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.