Research map / Architectures and efficiency / Pruning and efficient tuning
Structured Pruning of Neural Networks With Budget-Aware Regularization
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Knowledge Distillation from the unpruned teacher (ablation) | Budget-Aware Regularization pruning · CIFAR-10 | test accuracy: -1.37% at 2x; -0.40% at 16x without distillation | Helped |
| use a Sigmoid pruning schedule instead of a linear schedule (ablation) | Budget-Aware Regularization pruning · CIFAR-10 | test accuracy: -0.87% at 2x; -0.92% at 16x without the Sigmoid schedule | Helped |
| add mixed-connectivity blocks (ablation) | pruned residual networks · CIFAR-10, CIFAR-100, and Mio-TCD | effective pruned volume: 12% to 58% on CIFAR-10; 14% to 57% on CIFAR-100; 32% to 52% on MIO-TCD when removed | Helped |
| use a volume budget rather than a FLOP budget (ablation) | Budget-Aware Regularization pruning · CIFAR-10 | test accuracy versus volume pruning factor | Helped |
| use a FLOP budget rather than a volume budget (ablation) | Budget-Aware Regularization pruning · CIFAR-10 | test accuracy versus FLOP reduction factor | Helped |
| apply Budget-Aware Regularization pruning | WideResNet · CIFAR-10 and CIFAR-100 | test accuracy | Mixed |
| apply Budget-Aware Regularization pruning | WideResNet and ResNet50 · TinyImageNet and Mio-TCD | test accuracy | Mixed |
| apply Budget-Aware Regularization pruning at severe pruning | WideResNet and ResNet50 · CIFAR-10, CIFAR-100, TinyImageNet, and Mio-TCD | test accuracy: 16x | Helped |
| use a Sigmoid schedule instead of an exponential transition schedule | Budget-Aware Regularization pruning · CIFAR-10 and CIFAR-100 | test accuracy: 16x | Mixed |
Closest papers by meaning
- Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon (NeurIPS 2017 · similarity 0.93)
- Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework (ICML 2021 · similarity 0.92)
- Growing Efficient Deep Networks by Structured Continuous Sparsification (ICLR 2021 · similarity 0.91)
- Neural Pruning via Growing Regularization (ICLR 2021 · similarity 0.91)
- Pruning from Scratch (AAAI 2020 · similarity 0.91)
- UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.