Research map / Architectures and efficiency / Quantization
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STBLLM, a post-training structured N:M binarization framework that compresses LLM weights below 1 bit (down to 0.55 bits) using a Standardized Importance metric, layer-wise N:M allocation, and non-salient-aware trisection quantization, with a 2:4 sparse-tensor-core CUDA kernel.
Problem
Binarized (1-bit) LLMs still have substantial redundancy and remain too large for resource-constrained devices; the goal is to push LLM weight compression below 1 bit without fine-tuning.
Approach
Combine N:M structured sparsity with residual-approximation binarization: prune weights by a gradient-free Standardized Importance score (weight magnitude times input-feature norm), assign per-layer N:M ratios by layer L2-norm importance, and binarize the non-salient weights into sparse, intermediate and dense groups using a trisection search over break-points.
Key results
- On LLaMA-1-7B, 0.55-bit (4:8) STBLLM reaches WikiText2 perplexity 31.72 versus 688.73 for BiLLM at the same setting.
- On LLaMA-1-30B at 0.55 bits, STBLLM averages 51.78% zero-shot accuracy versus 43.72% for BiLLM (4:8).
- The custom 2:4 1-bit CUDA kernel achieves up to 17.85x speedup over ABQ-LLM's 2-bit implementation at sequence lengths 4096 and 8192 on RTX 4090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Standardized Importance (SI) pruning metric instead of Wanda (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 207.32 -> 31.72 | Helped |
| SI metric vs SparseGPT Hessian metric (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 32.82 -> 31.72 | Helped |
| Adaptive layer-wise N:M allocation instead of uniform allocation (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 80.36 -> 15.03 | Helped |
| Adaptive layer-wise allocation instead of sine-shaped allocation (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 67.78 -> 15.03 | Helped |
| Trisection non-salient-aware quantization instead of bell-shaped splitting (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 80.35 -> 15.03 | Helped |
| Group size 64 instead of 128 (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 31.72 -> 29.58 | Helped |
| Larger group size (1024) instead of 128 (ablation) | STBLLM on LLaMA-1-7B · Wikitext2 | perplexity: 31.72 -> 146.46 | Hurt |
| Adding STBLLM N:M structured binarization (4:8) on top of BiLLM binarization | LLaMA-1-7B · Wikitext2 perplexity at 0.55 bits | perplexity: 688.73 -> 31.72 | Helped |
| STBLLM vs BiLLM at 4:8 zero-shot | LLaMA-1-30B · 7 zero-shot tasks mean accuracy | mean accuracy (%): 43.72 -> 51.78 | Helped |
| STBLLM vs BiLLM at 4:8 on OPT and Mistral perplexity | OPT-1.3B to 30B and Mistral-7B · Wikitext2 perplexity at 0.55 bits | perplexity: Mistral-7B 189.73 -> 70.14; OPT-1.3B 106.99 -> 45.11 | Helped |
| Random sign flipping of non-salient binarized weights at increasing ratios (1%-16%) | binarized LLaMA-2-7B / LM · perplexity on Wikitext2 vs flip ratio | perplexity: 27.77 (top 1%) rising to 117.99 (17%) | Mixed |
Surprising findings
- Randomly flipping the signs of a fraction of non-salient binarized weights causes only minor performance drop, suggesting redundancy in 1-bit LLMs.
- Hessian-based importance is distorted by extreme weight values, motivating the gradient-free SI metric.
Limitations
- Does not support Mixture-of-Experts models.
- Does not support Mamba-based language models.
- Performance at the most extreme settings (e.g., LLaMA-3-8B at 0.55 bits, perplexity 253.76) remains high.
Experimental setup
Datasets: C4
Benchmarks: WikiText2 C4 PTB Winogrande OBQA HellaSwag BoolQ ARC-e ARC-c RTE
Metrics: perplexity zero-shot accuracy average bits runtime throughput memory
Baselines: BiLLM PB-LLM GPTQ RTN ABQ-LLM Wanda SparseGPT AWQ
Builds on: BiLLM Wanda SparseGPT DominoSearch GPTQ
Closest papers by meaning
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.89)
- PB-LLM: Partially Binarized Large Language Models (ICLR 2024 · similarity 0.89)
- ARB-LLM: Alternating Refined Binarizations for Large Language Models (ICLR 2025 · similarity 0.87)
- SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression (ICML 2025 · similarity 0.87)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.87)
- LittleBit: Ultra Low-Bit Quantization via Latent Factorization (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.