Research map / Architectures and efficiency / Pruning and efficient tuning
Effective Model Pruning : Measuring the Redundancy of Model Components
Yixuan Wang, Dan P. Guralnik, Saiedeh Akbari, Warren Dixon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| choose sparsity adaptively with EMP instead of fixed 50% for magnitude pruning (ablation) | magnitude pruning on LLaMA/LLaMA-2 · WikiText avg over 7 models | avg change in perplexity: +2.982 -> +0.752 at sparsity 50.00 -> 36.63 | Improved |
| choose sparsity adaptively with EMP instead of fixed 50% for magnitude pruning (ablation) | magnitude pruning on LLaMA/LLaMA-2 · zero-shot 7 tasks avg | avg change in accuracy (%): -2.60 -> -0.93 | Improved |
| apply EMP threshold to Wanda (ablation) | Wanda · LLaMA/LLaMA-2 avg | avg change in perplexity: +0.799 -> +0.678 at sparsity 50.00 -> 40.47 | No clear effect |
| EMP-Wanda on LLaMA-2 7B (ablation) | Wanda · LLaMA-2 7B WikiText | perplexity: 6.410 -> 6.513 | Hurt |
| prune with beta below 1 (beyond n_eff) (ablation) | EMP on FC models · MNIST / Fashion-MNIST | accuracy | Hurt |
| keep more than n_eff entries (beta above 1) (ablation) | EMP on FC models · MNIST / Fashion-MNIST | accuracy | No clear effect |
| apply EMP threshold to magnitude pruning of VGG16 (ablation) | VGG16 CIFAR-10 · CIFAR-10 | accuracy (%): 69.08 (90% fixed) -> 90.98 (EMP, 59.5% sparsity) | Improved |
| apply EMP to Taylor loss-change criterion (ablation) | VGG16 CIFAR-10 · CIFAR-10 | sparsity (%): accuracy 91.12 maintained at sparsity 2.1 | Mixed |
| EMP structured pruning at beta=1 on F-MNIST FC5 | FC5 · F-MNIST | accuracy (pp): 84.57 -> 86.44 | Improved |
| EMP structured pruning at beta=1 on CIFAR-10 VGG16 | VGG16 · CIFAR10 | accuracy (pp): 91.12 -> 90.69 | No clear effect |
| apply EMP to KAN node pruning | KAN [784,64,10] · MNIST | validation accuracy (%): 97.15 -> 94.36 | Hurt |
| EMP head pruning at beta=1 with Taylor criterion | GPT-2 · WikiText-2 | perplexity: 34.86 -> 35.20 | No clear effect |
Closest papers by meaning
- Entropy Induced Pruning Framework for Convolutional Neural Networks (AAAI 2024 · similarity 0.87)
- DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models (ICLR 2025 · similarity 0.86)
- Exploring Learning Complexity for Efficient Downstream Dataset Pruning (ICLR 2025 · similarity 0.86)
- Sparsest Models Elude Pruning: An Exposé of Pruning’s Current Capabilities (ICML 2024 · similarity 0.86)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.86)
- Effective pruning of web-scale datasets based on complexity of concept clusters (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
