Research map / Architectures and efficiency / Pruning and efficient tuning
Advancing Model Pruning via Bi-level Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prune with BiP instead of IMP | ResNet-18 · CIFAR-10 | sparsity of sparsest winning ticket (%): 87 -> 89 | Helped |
| prune with BiP instead of IMP | ResNet-20 · CIFAR-10 | sparsity of sparsest winning ticket (%): 74 -> 67 | Hurt |
| prune with BiP instead of IMP | ResNet-50 · ImageNet | sparsity of sparsest winning ticket (%): 80 -> 74 | Hurt |
| prune with BiP instead of IMP | ResNet-20 · CIFAR-100 | winning ticket existence: none -> 49% sparsity | Helped |
| prune with BiP instead of IMP for runtime | ResNet-18 · CIFAR-10 | pruning runtime (min): 620 -> 86 | Helped |
| remove implicit gradient term from BiP (ablation) | BiP · CIFAR-10 | test accuracy | Helped |
| add retraining/rewinding after BiP pruning (ablation) | BiP · CIFAR/others | test accuracy | No clear effect |
| increase lower-level SGD steps from 1 to 3 or 5 (ablation) | BiP · pruning | final test accuracy | No clear effect |
| train BiP longer than 100 epochs at high sparsity (ablation) | BiP · pruning | test accuracy | No clear effect |
| use different random batches for upper and lower levels instead of same batch (ablation) | BiP · pruning | test accuracy | Helped |
| reduce IMP retraining epochs per cycle to 20/40/80 (ablation) | IMP · pruning | accuracy | Hurt |
| increase lower-level regularization gamma too large or too small (ablation) | BiP · pruning | test accuracy | Mixed |
Closest papers by meaning
- Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models (ICML 2023 · similarity 0.91)
- Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? (ICLR 2023 · similarity 0.90)
- Winning the Lottery with Continuous Sparsification (NeurIPS 2020 · similarity 0.89)
- Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks (AAAI 2025 · similarity 0.88)
- No Free Prune: Information-Theoretic Barriers to Pruning at Initialization (ICML 2024 · similarity 0.87)
- Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.