Research map / Large language models / Language, safety and interpretability
Improving Generalization in LLM Structured Pruning via Function-Aware Neuron Grouping
Tao Yu, Yongqi An, Kuan Zhu, Guibo Zhu, Ming Tang, Jinqiao Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive sparsity allocation (ASA) to OBC baseline (ablation) | OBC · 7-task avg, 30% sparsity | avg zero-shot accuracy: 57.88 -> 59.17 | Improved |
| add shared neuron group retention (SNGR) (ablation) | OBC + ASA · 7-task avg, 30% sparsity | avg zero-shot accuracy: 59.17 -> 59.64 | Improved |
| add function-aware pruning (FAP) (ablation) | OBC + ASA + SNGR · 7-task avg, 30% sparsity | avg zero-shot accuracy: 59.64 -> 59.83 | Improved |
| add function-aware pruning (FAP) (ablation) | OBC + ASA + SNGR · 7-task avg, 40% sparsity | avg zero-shot accuracy: 54.38 -> 54.96 | Improved |
| function-based neuron grouping instead of random grouping (ablation) | O-FANG · 7-task avg, 40% sparsity | avg zero-shot accuracy: 53.94 -> 54.96 | Improved |
| function-based neuron grouping instead of random grouping (ablation) | O-FANG · 7-task avg, 30% sparsity | avg zero-shot accuracy: 59.20 -> 59.83 | Improved |
| semantic-distance softmax reweighting instead of uniform weights (ablation) | O-FANG · 7-task avg, 30% sparsity | avg zero-shot accuracy: 59.47 -> 59.83 | Improved |
| semantic-distance reweighting instead of Only-Matched cluster (ablation) | O-FANG · 7-task avg, 30% sparsity | avg zero-shot accuracy: 55.90 -> 59.83 | Improved |
| semantic-distance reweighting instead of Reverse weights (ablation) | O-FANG · 7-task avg, 40% sparsity | avg zero-shot accuracy: 52.44 -> 54.96 | Improved |
| functional-complexity sparsity allocation instead of Taylor-based (ablation) | O-FANG · LLaMA2-7B 30% sparsity | avg zero-shot accuracy: 58.85 -> 59.83 | Improved |
| functional-complexity sparsity allocation instead of Taylor-based (ablation) | O-FANG · LLaMA2-13B 40% sparsity | avg zero-shot accuracy: 52.17 -> 61.16 | Improved |
| WikiText calibration with FANG instead of Alpaca calibration with OBC+ASA | O-FANG · LLaMA2-7B 30% | perplexity: 10.24 -> 7.23 | Improved |
Closest papers by meaning
- Olica: Efficient Structured Pruning of Large Language Models without Retraining (ICML 2025 · similarity 0.89)
- SlimGPT: Layer-wise Structured Pruning for Large Language Models (NeurIPS 2024 · similarity 0.89)
- The Sparsity Whisperer (NeurIPS 2026 · similarity 0.88)
- Layer as Puzzle Pieces: Compressing Large Language Models through Layer Concatenation (NeurIPS 2025 · similarity 0.88)
- Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment (AAAI 2025 · similarity 0.87)
- Fluctuation-Based Adaptive Structured Pruning for Large Language Models (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
