Research map / Architectures and efficiency / Quantization
Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Basis Sharing, which compresses pretrained LLMs by SVD-decomposing concatenated weight matrices across adjacent layers into a shared basis plus per-layer coefficients, guided by Frobenius-loss analysis.
Problem
Reducing the memory footprint of LLMs for deployment; existing SVD-based compression treats each weight matrix in each layer independently, and prior cross-layer sharing requires training from scratch and hurts performance.
Approach
Represent weight matrices of several adjacent layers as a linear combination of a shared set of basis vectors (from SVD of the horizontally concatenated, activation-whitened weights) plus layer-specific coefficients, applied post-hoc to pretrained LLMs without retraining; select which matrix types (K, Q, V, Up, Gate; not Down or O) and which layers (adjacent pairs) to share based on Frobenius loss
Key results
- Under 20%-50% compression, Basis Sharing reduces perplexity by up to 25% versus SVD-LLM on generation tasks and improves average reasoning accuracy by at least 3% over state-of-the-art methods on LLaMA-7B.
- On OPT-6.7B and Mistral-7B at 20% compression, Basis Sharing achieves up to 25% PPL reduction relative to SVD-LLM on WikiText-2.
- At 50% compression on LLaMA-7B, Basis Sharing achieves 1.57x the throughput of the dense model on a single A100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Share bases across layers instead of compressing each weight matrix independently | Basis Sharing vs. SVD-LLM · OPT-6.7B and Mistral-7B, WikiText-2, 20% compression ratio | perplexity: up to 25% | Helped |
| Share bases across layers instead of using state-of-the-art SVD-based compression | Basis Sharing · Downstream reasoning tasks, compression ratios 20% to 50% | accuracy: up to 4% | Helped |
| Share bases across layers | LLaMA-7B · A100, batch size 512, sequence length 32, 50% compression ratio | throughput: 1.57 times | Helped |
| Share a basis matrix across layers 9 and 10 | LLaMA2-7B W_K · W_K, layers 9 and 10 | Frobenius loss: 66682.9 -> 61817.3 | Helped |
| Share a basis matrix across layers 9 and 10 | LLaMA2-7B W_O · W_O, layers 9 and 10 | Frobenius loss: 4355.1 + 4895.7 -> 10618.3 | Hurt |
| Use Alpaca rather than WikiText-2 to evaluate S (ablation) | LLaMA2-7B Basis Sharing · Common reasoning tasks, especially ARC_e at 50% compression ratio | accuracy: 11% | Helped |
| Use Alpaca rather than WikiText-2 to evaluate S (ablation) | LLaMA2-7B Basis Sharing · WinoG., 40% and 50% compression ratios | accuracy | Mixed |
| Group four or five consecutive layers rather than two (ablation) | LLaMA-7B Basis Sharing · Zero-shot, compression ratio lower than 30% | perplexity | Helped |
| Group two consecutive layers rather than four or five (ablation) | LLaMA-7B Basis Sharing · Zero-shot, compression ratio larger than 30% | perplexity | Helped |
| Apply LoRA fine-tuning (ablation) | LLaMA-7B Basis Sharing · WikiText-2, grouped-layer settings | perplexity | Helped |
| Group layers for Basis Sharing rather than use no basis sharing (# LAYERS = 1) (ablation) | LLaMA-7B Basis Sharing with LoRA fine-tuning · WikiText-2, grouped-layer settings including 32 layers | perplexity | Helped |
| Use full-parameter fine-tuning rather than LoRA fine-tuning (ablation) | LLaMA-7B Basis Sharing · WikiText-2, grouped-layer settings | perplexity | Helped |
Surprising findings
- Sharing bases for W_O across layers increases Frobenius loss, so it is excluded, while sharing for W_K, W_Q, W_V, W_Up, W_Gate reduces loss.
- Under 20% compression, Basis Sharing with a single layer group performs similarly to SVD-LLM, and the benefit mainly appears with grouping at larger compression ratios.
Limitations
- Compression error grows at 40%+ compression, requiring input-deviation updates for later layers.
- Performance is sensitive to the calibration dataset used to derive the whitening matrix (e.g., WikiText-2 vs Alpaca on ARC_e).
- Full parameter fine-tuning was run only for LLaMA-7B with a few group sizes due to computational cost.
Experimental setup
Datasets: WikiText-2 Alpaca PTB C4
Benchmarks: WikiText-2 PTB C4 OpenbookQA WinoGrande HellaSwag PIQA MathQA ARC-e ARC-c
Metrics: perplexity accuracy throughput Frobenius loss
Baselines: SVD-LLM ASVD FWSVD Dynamic Tying
Builds on: SVD-LLM ASVD FWSVD Dynamic Tying
Closest papers by meaning
- Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives (ICLR 2025 · similarity 0.91)
- SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression (ICLR 2025 · similarity 0.90)
- LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression (NeurIPS 2026 · similarity 0.90)
- MoE-SVD: Structured Mixture-of-Experts LLMs Compression via Singular Value Decomposition (ICML 2025 · similarity 0.89)
- Compressing Large Language Models using Low Rank and Low Precision Decomposition (NeurIPS 2024 · similarity 0.89)
- MoDeGPT: Modular Decomposition for Large Language Model Compression (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.