Research map / Architectures and efficiency / Quantization
ARB-LLM: Alternating Refined Binarizations for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ARB-LLM, a 1-bit post-training binarization method for LLMs that alternately refines mean, scaling factors and binary matrices, extended with calibration-aware (ARB-X) and row-column scaling (ARB-RC) variants plus a column-group bitmap.
Problem
Binary post-training quantization of LLM weights suffers from a distribution gap between binarized and full-precision weights and ignores column-wise deviations and calibration data, causing large perplexity degradation at 1-bit.
Approach
Alternately and iteratively refine the shifting mean, row scaling factors and binary matrices to minimize quantization error (ARB), then extend the objective with calibration-data Hessian-like statistics (ARB-X) and a column scaling factor (ARB-RC), and refine salient/non-salient partitioning via a column-group bitmap (CGB).
Key results
- ARB-LLM_RC reduces WikiText2 perplexity on LLaMA-7B to 14.03 at ~1.11 bits versus 49.79 for BiLLM.
- ARB-LLM_RC is reported as the first binary PTQ method to surpass same-size FP16 models on zero-shot QA average accuracy.
- Perplexity reduced by up to 68.7% relative to BiLLM on the OPT family without increasing weight bit-width.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add calibration-data updates to ARB (ablation) | ARB · LLaMA-7B, WikiText2 | perplexity: 22.67 -> 21.81 | Helped |
| Add row-column updates to ARB (ablation) | ARB · LLaMA-7B, WikiText2 | perplexity: 22.67 -> 14.03 | Helped |
| Add CGB to ARB-LLM_X (ablation) | ARB-LLM_X · LLaMA-7B, WikiText2 | perplexity: 26.29 -> 21.81 | Helped |
| Add CGB to ARB-LLM_RC (ablation) | ARB-LLM_RC · LLaMA-7B, WikiText2 | perplexity: 15.85 -> 14.03 | Helped |
| Use both column and group bitmaps rather than either bitmap alone (ablation) | ARB-LLM_RC · LLaMA-7B, WikiText2 | perplexity: 369.20 -> 14.03; 920.42 -> 14.03 | Helped |
| Increase ARB-LLM_X calibration set size from 64 to 128 (ablation) | ARB-LLM_X · LLaMA-7B, WikiText2 | perplexity: 24.79 -> 21.81 | Helped |
| Increase ARB-LLM_X calibration set size from 128 to 256 (ablation) | ARB-LLM_X · LLaMA-7B, WikiText2 | perplexity: 21.81 -> 21.88 | No clear effect |
| Increase ARB-LLM_X iterations from 1 to 15 (ablation) | ARB-LLM_X · LLaMA-7B, WikiText2 | perplexity: 22.59 / 21.12 / 21.81 | Mixed |
| Increase ARB-LLM_RC iterations from 1 to 15 (ablation) | ARB-LLM_RC · LLaMA-7B, WikiText2 | perplexity: 15.23 / 14.34 / 14.03 | Helped |
| Increase group count from 2 to 4 (ablation) | ARB-LLM_X · LLaMA-7B, WikiText2 | perplexity: 21.81 / 6.55 | Helped |
| Increase group count from 2 to 4 (ablation) | ARB-LLM_RC · LLaMA-7B, WikiText2 | perplexity: 14.03 / 12.77 | Helped |
| Use ARB-LLM_RC instead of BiLLM | LLaMA-7B · WikiText2 | perplexity: 49.79 -> 14.03 | Helped |
Surprising findings
- Row-column scaling without the alternating update fails to improve and can increase quantization error.
- ARB-RC with 1 iteration already outperforms BiLLM substantially, and the alternating update yields most gains at fewer iterations.
Limitations
- Binarization still incurs noticeable perplexity gap on some models (e.g., LLaMA-3 WikiText2 27.42 for ARB-RC at 7-8B)
- CGB increases storage and time overhead due to percentile search and extra scaling factors
- ARB-RC cannot easily incorporate calibration data due to parameter coupling
Experimental setup
Datasets: C4
Benchmarks: WikiText2 PTB C4 ARC-c ARC-e BoolQ Hellaswag OBQA PIQA Winogrande
Metrics: perplexity zero-shot accuracy memory GB time
Baselines: BiLLM PB-LLM GPTQ RTN
Builds on: BiLLM GPTQ PB-LLM
Closest papers by meaning
- PB-LLM: Partially Binarized Large Language Models (ICLR 2024 · similarity 0.95)
- BiLLM: Pushing the Limit of Post-Training Quantization for LLMs (ICML 2024 · similarity 0.94)
- QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.91)
- Treasures in Discarded Weights for LLM Quantization (AAAI 2025 · similarity 0.90)
- SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models (ICML 2025 · similarity 0.90)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.