Research map / Large language models / Reasoning and chain of thought
SKIM: Pruning Large Language Model Agents via Selective Knowledge Informed Masking
Moonseok Choi, Giung Nam, Jongwon Jeong, Minki Kang, Juho Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SKIM, a pruning method for LLM agents under grammar-constrained structured generation that accounts for the mismatch between constrained output distributions and standard unconstrained pruning criteria.
Problem
Compression methods for LLMs target unconstrained generation, but LLM agents are often deployed under grammar-constrained structured generation, where the output distribution is restricted to valid tokens and the logits that matter lie in a small set of positions.
Approach
Identifies a mismatch under structured generation: grammar constraints reduce the output distribution to valid tokens, so logits matter most at a small set of constrained positions; SKIM uses selective, knowledge-informed masking to prune the model accordingly, rather than using unconstrained pruning criteria.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add on-policy distillation phase (ablation) | SKIM · BFCL-v3 25% sparsity, Qwen3.5-2B | structured accuracy: 78.1 -> 80.8 | Improved |
| add category-aware feature distillation (ablation) | SKIM · BFCL-v3 25% sparsity, Qwen3.5-2B | structured accuracy: 79.1 -> 80.8 | Improved |
| add vanilla output-matching KD (beta 0.5) (ablation) | SKIM · BFCL-v3 25% sparsity, Qwen3.5-2B | structured accuracy: 80.8 -> 77.9 | Hurt |
| add vanilla KD (beta 0.5) (ablation) | SKIM · BFCL-v3 25% sparsity, Qwen3.5-2B | free-generation accuracy: 0.0 -> 46.4 | Improved |
| down-weight format positions (alpha_f 0.2 instead of 1.0 or 0) (ablation) | SKIM · API-Bank L1 25% sparsity, Qwen3.5-2B | structured accuracy | Improved |
| use tool-span rule labels instead of GPT-4 context labels (ablation) | SKIM · BFCL-v3, Qwen3.5-9B | structured accuracy: 85.5 -> 84.0 (25%), 77.6 -> 73.0 (50%) | Hurt |
| use tool-span rule labels instead of GPT-4 context labels (ablation) | SKIM · BFCL-v3, Qwen3.5-2B | structured accuracy: 80.8 -> 81.7 (25%), 64.5 -> 64.3 (50%) | No clear effect |
| label all prompt tokens as context (ablation) | SKIM · BFCL-v3, Qwen3.5-9B | structured accuracy: 85.5 -> 78.0 (25%) | Hurt |
| category-aware pruning and distillation vs uniform saliency plus vanilla KD | SKIM · Spider, Qwen3.5-2B 10% sparsity | component F1: 53.1 -> 56.4 | Improved |
| category-aware pruning and distillation vs uniform saliency plus vanilla KD | SKIM · GSM8K JSON, Qwen3.5-2B 10% sparsity | exact match: 21.0 -> 24.0 | Improved |
| SKIM instead of strongest baseline pruning at 50% sparsity | SKIM · BFCL-v3, Hammer2.1-3B | structured accuracy: 73.0 -> 82.8 | Improved |
| SKIM instead of baselines at 10% sparsity | SKIM · BFCL-v3, Qwen3.5-2B | structured accuracy: FLAP 83.8 vs SKIM 82.9 | No clear effect |
Closest papers by meaning
- Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression (ICML 2025 · similarity 0.90)
- Compact Language Models via Pruning and Knowledge Distillation (NeurIPS 2024 · similarity 0.89)
- You Only Prune Once: Designing Calibration-Free Model Compression With Policy Learning (ICLR 2025 · similarity 0.88)
- Scaling up Masked Diffusion Models on Text (ICLR 2025 · similarity 0.88)
- Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing (ICLR 2025 · similarity 0.87)
- Discovering Sparsity Allocation for Layer-wise Pruning of Large Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
