AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference

Hao Ma, Melis Ilayda Bal, Liang Zhang, Bingcong Li, Niao He, Melanie Zeilinger, Michael Muehlebach

ICML 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Low-rank compression and merging · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use SALAAD instead of full-rank trainingLLaMA-based 1B · C4perplexity: 15.56 -> 14.74Improved
use SALAAD instead of SLTrain at similar budgetLLaMA-based 350M · C4perplexity: 19.42 -> 18.04 (L+S)Improved
use SALAAD instead of LOSTLLaMA-based 350M, HPA-compressed · C4perplexity: 18.95 -> 19.70Hurt
add HPA compression of SALAAD model to 646MLLaMA-based 1B · C4perplexity: 14.72 -> 15.07 (L+S to eL+eS)Mixed
include embedding layer in SALAAD training (ablation)LLaMA-based 350M · C4training lossNo clear effect
include LM head in SLR induction (ablation)SALAAD · LLaMA-basedSLR structureHurt
apply SLR induction at training time instead of post-hoc RPCA on vanillaHPA elastic deployment · C4, 60M-1Bperplexity vs parametersImproved
assign larger share of budget to low-rank component (kappa > 0.5) (ablation)HPA allocation · C4, 130M/350M/1BperplexityImproved
increase delta_beta from 0.003 to 0.1 (ablation)SALAAD · C4, LLaMA 350Mperplexity (X): 18.62 -> 19.97Hurt
increase delta_alpha from 0.08 to 0.2 (ablation)SALAAD · C4, LLaMA 350Mperplexity (X): 18.25 -> 18.97Hurt
increase rho from 5e-8 to 1e-7 (ablation)SALAAD · C4, LLaMA 350Mperplexity (X): 18.45 -> 19.53Hurt
compress SALAAD 1B to 646MSALAAD · zero-shot downstream, 1Baccuracy: COPA 71.0 -> 69.0; BoolQ 54.0 -> 52.0No clear effect

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy