Research map / Architectures and efficiency / Quantization
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SeedLM compresses LLM weight blocks into LFSR pseudo-random generator seeds plus few quantized coefficients, achieving data-free 3/4-bit post-training compression with near-FP16 zero-shot accuracy and FPGA speedups.
Problem
Post-training LLM weight compression for deployment is memory-bound; state-of-the-art low-bit methods require calibration data and per-layer tuning, and memory reads dominate inference cost.
Approach
Represent each block of C weights as a linear combination of P columns from a pseudo-random matrix generated on the fly by an LFSR seeded with a searched K-bit seed; store only the seed and 4-bit quantized coefficients, trading compute for fewer memory reads, with no calibration data.
Key results
- SeedLM 4-bit on Llama 3 70B retains about 97.9% of zero-shot accuracy on average (mean 78.06 vs 79.51 FP16), while AWQ 4-bit gets 72.34.
- At 3-bit on Llama 3 70B, SeedLM mean accuracy is 74.68 versus 61.35 for AWQ and 32.95 for OmniQuant.
- FPGA prototype achieves 3.67-3.98x speedup over the FP16 reference design for 512-2048 matrix sizes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 2 7B · zero-shot tasks | Mean accuracy: 67.30 -> 67.57 | Helped |
| Use SeedLM 3-bit instead of QuIP# 3-bit | Llama 2 7B · zero-shot tasks | Mean accuracy: 55.85 -> 64.48 | Helped |
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 2 13B · zero-shot tasks | Mean accuracy: 71.14 -> 71.46 | Helped |
| Use SeedLM 3-bit instead of QuIP# 3-bit | Llama 2 13B · zero-shot tasks | Mean accuracy: 68.44 -> 68.58 | Helped |
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 2 70B · zero-shot tasks | Mean accuracy: 76.27 -> 75.90 | Hurt |
| Use SeedLM 3-bit instead of AWQ 3-bit | Llama 2 70B · zero-shot tasks | Mean accuracy: 73.91 -> 73.83 | Hurt |
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 3 8B · zero-shot tasks | Mean accuracy: 71.51 -> 71.31 | Hurt |
| Use SeedLM 3-bit instead of QuIP# 3-bit | Llama 3 8B · zero-shot tasks | Mean accuracy: 62.84 -> 62.79 | Hurt |
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 3 70B · zero-shot tasks | Mean accuracy: 72.34 -> 78.06 | Helped |
| Use SeedLM 3-bit instead of AWQ 3-bit | Llama 3 70B · zero-shot tasks | Mean accuracy: 61.35 -> 74.68 | Helped |
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 2 13B · WikiText-2 test | Perplexity: 5.1 -> 5.1 | No clear effect |
| Use SeedLM 4-bit instead of AWQ 4-bit | Llama 3 70B · WikiText-2 test | Perplexity: 4.7 -> 3.8 | Helped |
Surprising findings
- OmniQuant 4-bit collapses to near-chance accuracy on Llama 3 70B (mean 33.45) despite calibration data.
- Llama 3 is markedly more sensitive to compression than Llama 2, which the authors attribute to its much larger training corpus.
Limitations
- Evaluation of speedup is limited to an FPGA prototype for matrix-vector multiplication, not end-to-end LLM inference.
- Llama 3 shows larger accuracy drops at 3-bit (mean 62.79 for 8B vs 72.26 baseline), so retention varies by model.
- Reconstruction-error design-space exploration assumes Gaussian weights.
Experimental setup
Datasets: WikiText-2
Benchmarks: WikiText-2 ARC-Easy ARC-Challenge HellaSwag WinoGrande BoolQ
Metrics: perplexity accuracy retained accuracy speedup cycles
Baselines: AWQ OmniQuant QuIP# GPTQ
Builds on: NOLA PRANC LoRA
Closest papers by meaning
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.90)
- SqueezeLLM: Dense-and-Sparse Quantization (ICML 2024 · similarity 0.90)
- Search for Efficient Large Language Models (NeurIPS 2024 · similarity 0.89)
- Compressing LLMs: The Truth is Rarely Pure and Never Simple (ICLR 2024 · similarity 0.89)
- Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs (ICML 2024 · similarity 0.89)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.