Research map / Architectures and efficiency / Quantization
1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization
Sohir Maskey, Constantin Eichenberg, Johannes Messner, Douglas Orr
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use k-means (nonlinear) weight format instead of uniform integer under QAT (ablation) | QAT of Llama-style LLMs (0.8B-3.9B) · Nemotron-CC pretraining loss scaling law | loss | Improved |
| k-means instead of uniform at Pw <= 4.25 bits (ablation) | QAT scaling law · memory-matched capacity per bit | effective capacity per bit g(Pw) | Improved |
| reduce weight precision and reinvest memory in more parameters (fixed memory budget) | QAT k-means models · scaling law prediction | loss | Improved |
| 31B 1-bit k-means model instead of 4B bf16 at equal weight memory | 4B bf16 baseline · MMLU | accuracy: 33.21 -> 51.61 | Improved |
| 31B 1-bit instead of 4B bf16 | 4B bf16 baseline · HumanEval Instruct | pass@1: 26.22 -> 42.68 | Improved |
| 31B 1-bit instead of 4B bf16 | 4B bf16 baseline · AidanBench | coherence score: 88.08 -> 167.83 | Improved |
| 31B 1-bit instead of 12B 4-bit on GSM8K | 12B 4-bit k-means · GSM8K (5-shot) | accuracy: 48.52 -> 45.26 | Hurt |
| 31B 1-bit instead of 12B 4-bit on HellaSwag/IFEval | 12B 4-bit k-means · HellaSwag | accuracy: 55.41 -> 54.70 | No clear effect |
| use 4-bit k-means LUT kernel instead of bf16 at batch size 1 | matvec kernel on L40S · 1x8192 by 8192x8192 matvec | time: 175.6 us -> 49.5 us (3.7x) | Improved |
| 1-bit k-means LUT kernel instead of bf16 | matvec kernel on L40S · 1x8192 by 8192x8192 matvec | time: 175.6 us -> 24.0 us (7.6x) | Improved |
| 1-bit 31B model instead of 4-bit 12B in generation throughput | 12B 4-bit k-means · token generation on L40S | tokens/s: 79.3 -> 60.7 (12B 4.25-bit vs 31B 1.25-bit) | Hurt |
| increase batch size for quantized kernels (ablation) | weight-quantized inference · kernel speedup | speedup: no speedup observed by batch size 256 | Hurt |
Closest papers by meaning
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.90)
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.89)
- QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead (AAAI 2025 · similarity 0.89)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.88)
- HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs (NeurIPS 2026 · similarity 0.88)
- QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
