Research map / Architectures and efficiency / Pruning and efficient tuning
SubspacePath Pruner: Inference-time Pruning via Probe-based Representation–Parameter Coupling
Gong Zhiren, Yikun Hou, Fan Wu, CHE WANG, Fuyao Zhang, Tiantong Wu, Yurong Hao, Jiaming Zhang, Yiyang Duan, Tiantong Wang, Fei Huang, Chau Yuen and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DBS axis selection (vs random axes) (ablation) | SubspacePath Pruner · XDomainBench Selected (LLaMA-2-13B) | recall (%): 1.8 -> 34.7 | Improved |
| add DBS axis selection (vs random axes) (ablation) | SubspacePath Pruner · XDomainBench OOD | recall (%): 1.4 -> 30.4 | Improved |
| add DBS axis selection (vs random axes) (ablation) | SubspacePath Pruner · XDomainBench Cross | recall (%): 0.9 -> 22.9 | Improved |
| add whitelist heads (ablation) | SubspacePath Pruner · XDomainBench OOD | recall (%): 0.4 -> 30.4 | Improved |
| add whitelist heads (ablation) | SubspacePath Pruner · XDomainBench Selected | recall (%): 22.4 -> 34.7 | Improved |
| add multi-domain mixing (ablation) | SubspacePath Pruner · XDomainBench Selected | recall (%): 29.5 -> 34.7 | Improved |
| add multi-domain mixing (ablation) | SubspacePath Pruner · XDomainBench OOD | recall (%): 23.5 -> 30.4 | Improved |
| moderate pruning with SubspacePath vs dense | LLaMA-2-13B-chat · XDomainBench Selected | recall (%): 29.6 -> 43.00 | Improved |
| moderate pruning with SubspacePath vs dense | LLaMA-2-13B-chat · Natural Questions | recall (%): 30.25 -> 33.66 | Improved |
| moderate pruning with SubspacePath vs dense | LLaMA-2-13B-chat · CommonsenseQA | recall (%): 22.27 -> 19.43 | Hurt |
| aggressive pruning with SubspacePath vs dense | LLaMA-2-13B-chat · XDomainBench Cross | recall (%): 18.4 -> 22.90 | Improved |
| SubspacePath on Qwen2.5-14B vs Mixtral top-2 | Qwen2.5-14B · XDomainBench Cross | recall (%): 24.0 -> 31.3 | Improved |
Closest papers by meaning
- Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing (ICLR 2025 · similarity 0.88)
- DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models (ICLR 2025 · similarity 0.86)
- POP: Online Structural Pruning Enables Efficient Inference of Large Foundation Models (NeurIPS 2026 · similarity 0.86)
- SkipGPT: Each Token is One of a Kind (ICML 2025 · similarity 0.85)
- Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization (NeurIPS 2025 · similarity 0.85)
- The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
