Research map / Architectures and efficiency / Quantization
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
BiLLM compresses pretrained LLM weights to about 1.08-1.11 bits via post-training binarization, using structured salient-column residual binarization and an optimal-break-point split for non-salient weights.
Problem
Post-training quantization of LLMs collapses at ultra-low bit-widths (≤3 bits), and prior binarization methods like PB-LLM need high-precision outliers that push average bit-width well above 1 bit.
Approach
Select salient weights structurally by column using a Hessian-based sensitivity metric and binarize them with a residual (second-order) approximation; split the bell-shaped non-salient weights at a searched break-point into concentrated and sparse groups, binarizing each with its own scale, combined with block-wise OBC/GPTQ error compensation.
Key results
- BiLLM with 1.08-bit weights achieves 8.41 WikiText2 perplexity on LLaMA2-70B and 8.49 on LLaMA-65B, below the FP16 OPT-66B (9.34).
- On OPT-family models, BiLLM at 1.11 bits gives 12.71 perplexity on OPT-30B versus 25.14 for PB-LLM at 1.7 bits and 15.71 for 2-bit GPTQ.
- Binarizing a 7B LLM takes about 30 minutes on a single A100 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BiLLM instead of PB-LLM at the reported lower weight bit-width | OPT-30B · WikiText2 | perplexity: 25.14 -> 12.71 | Helped |
| use BiLLM instead of 2-bit GPTQ | LLaMA-7B · WikiText2 | perplexity: 152.31 -> 35.04 | Helped |
| use BiLLM instead of PB-LLM | LLaMA2-70B · WikiText2 | perplexity: 28.37 -> 8.41 | Helped |
| use BiLLM instead of PB-LLM | Vicuna-7B · PTB | perplexity: 477.52 -> 332.17 | Helped |
| use BiLLM instead of PB-LLM | LLaMA-7B · PIQA | accuracy: 54.6 -> 61.2 | Helped |
| use BiLLM instead of PB-LLM | LLaMA2-7B · ARC-c | accuracy: 25.0 -> 24.4 | Hurt |
| add salient-weight residual approximation (ablation) | BiLLM · OPT-6.7B and LLaMA-7B | perplexity | Helped |
| add optimal splitting for non-salient weights (ablation) | BiLLM · OPT-6.7B and LLaMA-7B | perplexity | Helped |
| set block size from 128 to 32 (ablation) | BiLLM on LLaMA-7B · WikiText2 | perplexity: 35.04 -> 17.56 | Helped |
| set block size from 128 to 512 (ablation) | BiLLM on LLaMA-7B · WikiText2 | perplexity: 35.04 -> 74.14 | Hurt |
Surprising findings
- 1.08-bit BiLLM on LLaMA2-70B and LLaMA-65B outperforms the full-precision OPT-66B on WikiText2 perplexity.
- Ablations show OPT-6.7B is more sensitive to the non-salient splitting while LLaMA-7B is more sensitive to residual approximation of salient weights.
Limitations
- Binarized GEMM is hard to implement directly due to fine-grained grouping, so real speedups are not demonstrated.
- Average bit-width excludes an extra 1-bit group identifier for the split non-salient weights, which the paper argues does not affect compute.
- Performance on zero-shot tasks is often near chance for some models and tasks, and the zero-shot gains are modest in absolute terms.
Experimental setup
Datasets: WikiText2 PTB C4
Benchmarks: WikiText2 PTB C4 PIQA BoolQ OBQA Winogrande ARC-e ARC-c HellaSwag
Metrics: perplexity accuracy average weight bits memory occupancy
Baselines: RTN GPTQ PB-LLM
Builds on: GPTQ OBC PB-LLM XNOR-Net
Closest papers by meaning
- ARB-LLM: Alternating Refined Binarizations for Large Language Models (ICLR 2025 · similarity 0.94)
- PB-LLM: Partially Binarized Large Language Models (ICLR 2024 · similarity 0.94)
- QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.92)
- SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models (ICML 2025 · similarity 0.92)
- Treasures in Discarded Weights for LLM Quantization (AAAI 2025 · similarity 0.92)
- QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.