Research map / Architectures and efficiency / Pruning and efficient tuning
Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks
Kaixin Xu, Zhe Wang, Xue Geng, Min Wu, Xiaoli Li, Weisi Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly optimize layerwise sparsity via additivity-based dynamic programming instead of single-layer optimization (ablation) | layer-adaptive weight pruning · ImageNet VGG-16-BN | Top-1 accuracy (%): 62.50 -> 69.01 | Improved |
| joint optimization vs single-layer on ResNet-50 (ablation) | layer-adaptive weight pruning · ImageNet ResNet-50 | Top-1 accuracy (%): 74.89 -> 75.59 | Improved |
| add outlier filtering of rate-distortion curves (ablation) | RD-based pruning · CIFAR-10 ResNet-32 (89% sparsity) | Top-1 accuracy (%): 91.15 -> 91.51 | Improved |
| add worst case sampling for distortion (ablation) | RD-based pruning · CIFAR-10 ResNet-32 (89% sparsity) | Top-1 accuracy (%): 91.15 -> 91.31 | Improved |
| add worst case sampling for distortion (ablation) | RD-based pruning · ImageNet ResNet-50 one-shot (60% sparsity) | Top-1 accuracy (%): 31.22 -> 31.22 | No clear effect |
| add both WCS and OF (ablation) | RD-based pruning · CIFAR-10 ResNet-32 (89% sparsity) | Top-1 accuracy (%): 91.15 -> 92.3 | Improved |
| joint optimization vs LAMP at high sparsity | iterative pruning · ImageNet ResNet-50 (~6% FLOPs) | Top-1 accuracy (%): 42.54 -> 67.91 | Improved |
| joint optimization vs LAMP on CIFAR-10 ResNet-32 at ~3% FLOPs | iterative pruning · CIFAR-10 ResNet-32 | Top-1 accuracy (%): 83.66 -> 90.83 | Improved |
| use white-noise data instead of real calibration data for RD curves | RD-based pruning · ImageNet ResNet-50 zero-data (50% sparsity) | Top-1 accuracy (%): 75.13 (zero-data, 1.01 drop) | No clear effect |
| joint optimization vs LAMP on ImageNet ResNet-50 at 95.5% sparsity | iterative pruning · ImageNet ResNet-50 | Top-1 accuracy (%): 66.04 -> 66.06 | No clear effect |
| use PGMPF with better dense baseline vs ours on VGG-16 CIFAR-10 | iterative pruning · CIFAR-10 VGG-16 (~33% FLOPs) | Top-1 accuracy (%): 93.6 vs 92.76 | Hurt |
Closest papers by meaning
- Efficient Distortion-Minimized Layerwise Pruning (TPAMI 2025 · similarity 0.93)
- A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers (ECCV 2018 · similarity 0.93)
- Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon (NeurIPS 2017 · similarity 0.91)
- Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework (ICML 2021 · similarity 0.91)
- Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining (ICLR 2022 · similarity 0.91)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
