Research map / Architectures and efficiency / Pruning and efficient tuning
ICP: Immediate Compensation Pruning for Mid-to-high Sparsity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICP, a one-shot pruning method that adds sliding-window inter-block error compensation on top of SparseGPT plus sparsity rearrangement, improving mid-to-high sparsity pruning of LLMs (OPT, Llama 2-7B) and SAM models without extra peak GPU memory.
Problem
One-shot pruning methods such as SparseGPT and Wanda perform well at low sparsity but degrade sharply at mid-to-high sparsity (50-70%), and full fine-tuning for compensation is infeasible for ~7B models on a single consumer GPU.
Approach
Prune blocks sequentially in a two-block sliding window: after pruning block i, block i+1 is updated via a short distillation-like regression to compensate for the pruning-induced output error, keeping only one block in memory; sparsity is redistributed across blocks (inter-block) and across layers within a block (intra-block) using priors.
Key results
- On OPT-6.7B at 70% unstructured sparsity, ICP reaches 17.82 WikiText perplexity versus 20.48 for SparseGPT and 159.2 for Wanda.
- On Llama 2-7B at 70% sparsity, ICP averages 46.94% zero-shot accuracy versus 41.79% for SparseGPT and 34.34% for Wanda.
- ICP keeps peak memory at 7.8 GB (same as or below SparseGPT's 8.3 GB) on a single RTX 3090, but pruning time is longer (1551 s for 1 epoch vs 1004 s for SparseGPT on OPT-6.7B).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ICP instead of SparseGPT | OPT-1.3B pruning · Wikitext-v2, 70% sparsity | perplexity: 51.75 -> 36.78 | Helped |
| use ICP instead of SparseGPT | OPT-1.3B pruning · PTB, 70% sparsity | perplexity: 76.91 -> 54.33 | Helped |
| use ICP instead of Wanda | Llama 2-7B pruning · Zero-shot tasks, 50% sparsity | Mean accuracy: 56.22 -> 56.96 | Helped |
| use ICP instead of SparseGPT | Llama 2-7B pruning · Zero-shot tasks, 70% sparsity | Mean accuracy: 41.79 -> 46.94 | Helped |
| use ICP instead of SparseGPT | SAM-H pruning · SA-1B, 70% sparsity | IoU: 65.69 -> 72.85 | Helped |
| use ICP instead of SparseGPT | SAM-H pruning · COCO, 70% sparsity | IoU: 66.70 -> 69.43 | Helped |
| use one compensation epoch instead of 12 (ablation) | ICP compensation · OPT-125M and SAM-B pruning | model performance | No clear effect |
| increase calibration set size beyond 128 samples (ablation) | ICP calibration · OPT-125M and SAM-B pruning | model performance | No clear effect |
| tune inter-block sparsity rearrangement parameter α (ablation) | ICP · OPT-125M and SAM-B pruning | perplexity / IoU | Helped |
| tune intra-block sparsity rearrangement parameter β (ablation) | ICP · OPT-125M pruning | perplexity | Helped |
| tune intra-block sparsity rearrangement parameter β (ablation) | ICP · SAM-B pruning | IoU | No clear effect |
Surprising findings
- Wanda underperforms SparseGPT on SAM vision models, which the authors attribute to language-specific priors in Wanda's importance metric.
Limitations
- Hyperparameters alpha and beta must be manually tuned and are not automatically set.
- Not tested on multimodal models.
- Pruning time is longer than SparseGPT and Wanda, especially with 12 compensation epochs.
Experimental setup
Datasets: C4 SA-1B WikiText-v2 PTB COCO BoolQ RTE HellaSwag WinoGrande ARC Easy ARC Challenge OpenBookQA
Benchmarks: WikiText-v2 PTB BoolQ RTE HellaSwag WinoGrande ARC Easy ARC Challenge OpenBookQA SA-1B COCO
Metrics: perplexity accuracy IoU peak memory usage pruning time
Baselines: Magnitude SparseGPT Wanda LLM-Pruner PruneZero
Builds on: SparseGPT Wanda Optimal Brain Surgeon
Closest papers by meaning
- CoreMatching: A Co-adaptive Sparse Inference Framework with Token and Neuron Pruning for Comprehensive Acceleration of Vision-Language Models (ICML 2025 · similarity 0.87)
- ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models (ICLR 2024 · similarity 0.86)
- DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs (NeurIPS 2025 · similarity 0.86)
- OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization (ICML 2024 · similarity 0.86)
- MDP: Multidimensional Vision Model Pruning with Latency Constraint (CVPR 2025 · similarity 0.86)
- Resource-Efficient Transformer Pruning for Finetuning of Large Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.