Research map / Architectures and efficiency / Pruning and efficient tuning
Neuron-level Structured Pruning using Polarization Regularizer
Tao Zhuang, Zhixuan Zhang, Yuheng Huang, Xiaoyi Zeng, Kai Shuang, Xiang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace L1 regularizer on BN scaling factors with polarization regularizer | network slimming ResNet-56 · CIFAR-10 | pruned accuracy (%): 93.27 -> 93.83 | Improved |
| replace L1 regularizer with polarization regularizer | network slimming ResNet-56 · CIFAR-100 | pruned accuracy (%): 71.40 -> 72.46 | Improved |
| replace L1 regularizer with polarization regularizer | network slimming VGG-16 · CIFAR-100 | pruned accuracy (%): 74.20 -> 74.25 | No clear effect |
| replace L1 regularizer with polarization regularizer | network slimming ResNet-50 · ImageNet | pruned top-1 accuracy (%): 74.88 -> 75.63 | Improved |
| replace L1 regularizer with polarization regularizer at ~70% FLOPs reduction | network slimming ResNet-50 · ImageNet | pruned top-1 accuracy (%): 70.61 -> 74.15 | Improved |
| replace L1 regularizer with polarization regularizer at ~57% FLOPs reduction | network slimming MobileNet v2 · ImageNet | pruned top-1 accuracy (%): 65.65 -> 67.49 | Improved |
| polarization regularizer vs uniform channel scaling at ~70% FLOPs reduction | ResNet-56 · CIFAR-10 | pruned accuracy (%): 92.25 -> 92.63 | Improved |
| polarization regularizer vs DeepHoyer at ~71% FLOPs reduction | ResNet-56 · CIFAR-10 | pruned accuracy (%): 91.26 -> 92.63 | Improved |
| increase hyper-parameter t of polarization regularizer (ablation) | polarization pruning · CIFAR-10 | FLOPs reduction / accuracy drop | Mixed |
| increase hyper-parameter lambda (ablation) | polarization pruning · CIFAR-10 | FLOPs reduction / accuracy drop | Hurt |
Closest papers by meaning
- Neuron Merging: Compensating for Pruned Neurons (NeurIPS 2020 · similarity 0.88)
- Pruning neural networks without any data by iteratively conserving synaptic flow (NeurIPS 2020 · similarity 0.87)
- Data-Efficient Structured Pruning via Submodular Optimization (NeurIPS 2022 · similarity 0.87)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.86)
- Towards Meta-Pruning via Optimal Transport (ICLR 2024 · similarity 0.86)
- SequentialAttention++ for Block Sparsification: Differentiable Pruning Meets Combinatorial Optimization (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
