Research map / Architectures and efficiency / Pruning and efficient tuning
SEAP: Sparse Expert Activation Pruning Unlocks the Brainpower of Large Language Models
Xun Liang, Hanyu Wang, Huayi Lai, Simin Niu, Shichao Song, Jiawei Yang, Jihao Zhao, Feiyu Xiong, Bo Tang, Zhiyu Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SEAP (sF) instead of FLAP at 20% pruning | LLaMA2-7B · 9 zero-shot tasks | accuracy avg (%): 52.81 -> 58.52 | Improved |
| use SEAP (sF) instead of FLAP at 50% pruning | LLaMA2-7B · 9 zero-shot tasks | accuracy avg (%): 35.26 -> 46.98 | Improved |
| use SEAP (sF) instead of FLAP at 50% pruning | LLaMA2-13B · 9 zero-shot tasks | accuracy avg (%): 39.68 -> 50.34 | Improved |
| use SEAP (sF) instead of FLAP at 20% pruning | LLaMA2-7B · GSM8K | accuracy (%): 1.81 -> 8.87 | Improved |
| use SEAP (sF) instead of FLAP at 20% pruning | LLaMA2-7B · MBPP | pass@1 (%): 1.20 -> 16.60 | Improved |
| add retention-based sparsity instead of uniform layerwise (ablation) | SEAP (sF) on LLaMA2-7B · 9 zero-shot tasks, 20% pruning | accuracy avg (%): 56.54 -> 58.52 | Improved |
| add retention-based sparsity instead of uniform layerwise (ablation) | SEAP (sF) on LLaMA2-7B · 9 zero-shot tasks, 50% pruning | accuracy avg (%): 43.17 -> 46.98 | Improved |
| add retention-based sparsity instead of adaptive layerwise (ablation) | FLAP on LLaMA2-7B · 9 zero-shot tasks, 50% pruning | accuracy avg (%): 35.26 -> 38.88 | Improved |
| add retention-based sparsity instead of uniform layerwise (ablation) | WandA-sp on LLaMA2-7B · 9 zero-shot tasks, 50% pruning | accuracy avg (%): 32.87 -> 34.50 | Improved |
| use WandA score (sW) instead of FLAP score (sF) | SEAP on LLaMA2-7B · 9 zero-shot tasks, 20% pruning | accuracy avg (%): 58.52 -> 57.88 | No clear effect |
| use SEAP instead of FLAP for inference speed at 20% pruning | LLaMA2-7B · inference throughput | tokens/s: 38.90 -> 37.32 | Hurt |
| use SEAP instead of WandA for inference speed at 50% pruning | LLaMA2-7B · inference throughput | tokens/s: 31.24 -> 47.10 | Improved |
Closest papers by meaning
- Sparsing Law: Towards Large Language Models with Greater Activation Sparsity (ICML 2025 · similarity 0.89)
- The Sparsity Whisperer (NeurIPS 2026 · similarity 0.89)
- Learn To be Efficient: Build Structured Sparsity in Large Language Models (NeurIPS 2024 · similarity 0.88)
- R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference (ICLR 2025 · similarity 0.88)
- Exploring Activation Patterns of Parameters in Language Models (AAAI 2025 · similarity 0.88)
- ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
