Research map / Architectures and efficiency / Pruning and efficient tuning
ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorization
Lawrence Liu, Alexander Liu, Mengdi Wang, Tuo Zhao, Lin F. Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| wrap 2:4 sparse core with block-diagonal matrices and optimize by block coordinate descent (ARMOR vs NoWag-P init) (ablation) | NoWag-P 2:4 · GSM8K Qwen2.5-7B | accuracy (%): 28.28 -> 53.28 | Improved |
| ARMOR vs NoWag-P (ablation) | NoWag-P 2:4 · MMLU Qwen2.5-7B | accuracy (%): 53.51 -> 65.56 | Improved |
| ARMOR vs NoWag-P (ablation) | NoWag-P 2:4 · Wikitext2 Llama-2-13B | perplexity: 8.28 -> 6.37 | Improved |
| ARMOR vs NoWag-P (ablation) | NoWag-P 2:4 · Wikitext2 Llama-3-8B | perplexity: 24.0 -> 10.10 | Improved |
| ARMOR on 2:4 Qwen2.5-32B GPQA vs dense | ARMOR · GPQA Qwen2.5-32B | accuracy (%): 30.36 (SparseGPT) -> 39.51 (dense 38.84) | Improved |
| apply ARMOR to other sparsity patterns (4:8, 5:8, 6:8, 50% unstructured) | NoWag-P · Wikitext2 Llama-2-7B | perplexity: 4:8: 8.04 -> 6.74; 50%: 6.37 -> 6.13 | Improved |
| increase block diagonal block size from 1 to 128 (ablation) | ARMOR · C4 perplexity Llama-2/3 | relative perplexity | Improved |
| minimize proxy loss longer (more optimization iterations) (ablation) | ARMOR · C4 perplexity Llama-2-7B | perplexity | Improved |
| ARMOR vs SparseGPT+DenoiseRotator at context 2048 | ARMOR · Wikitext2 Llama-3-8B | perplexity: 10.01 -> 11.27 | Hurt |
| ARMOR block-diagonal wrappers vs naive 2:4 inference | naive 2:4 · generation throughput Qwen2.5-7B | tokens/s: 5430 -> 5090 | Hurt |
Later papers that compared with this method
1 later accepted paper reports ARMOR as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Resting Neurons, Active Insights: Robustifying Activation Sparsity in LLMs via Spontaneity (ICML 2026 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization (NeurIPS 2025 · similarity 0.91)
- Targeted Low-rank Refinement: Enhancing Sparse Language Models with Precision (ICML 2025 · similarity 0.91)
- ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models (NeurIPS 2024 · similarity 0.91)
- Pruning Large Language Models with Semi-Structural Adaptive Sparse Training (AAAI 2025 · similarity 0.91)
- AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models (NeurIPS 2024 · similarity 0.90)
- An Efficient Pruner for Large Language Model with Theoretical Guarantee (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
