AI Paper Atlas

Research map / Architectures and efficiency / Quantization

1-Bit Wonder: Improving QAT Performance in the Low-Bit Regime through K-Means Quantization

Sohir Maskey, Constantin Eichenberg, Johannes Messner, Douglas Orr

ICML 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use k-means (nonlinear) weight format instead of uniform integer under QAT (ablation)QAT of Llama-style LLMs (0.8B-3.9B) · Nemotron-CC pretraining loss scaling lawlossImproved
k-means instead of uniform at Pw <= 4.25 bits (ablation)QAT scaling law · memory-matched capacity per biteffective capacity per bit g(Pw)Improved
reduce weight precision and reinvest memory in more parameters (fixed memory budget)QAT k-means models · scaling law predictionlossImproved
31B 1-bit k-means model instead of 4B bf16 at equal weight memory4B bf16 baseline · MMLUaccuracy: 33.21 -> 51.61Improved
31B 1-bit instead of 4B bf164B bf16 baseline · HumanEval Instructpass@1: 26.22 -> 42.68Improved
31B 1-bit instead of 4B bf164B bf16 baseline · AidanBenchcoherence score: 88.08 -> 167.83Improved
31B 1-bit instead of 12B 4-bit on GSM8K12B 4-bit k-means · GSM8K (5-shot)accuracy: 48.52 -> 45.26Hurt
31B 1-bit instead of 12B 4-bit on HellaSwag/IFEval12B 4-bit k-means · HellaSwagaccuracy: 55.41 -> 54.70No clear effect
use 4-bit k-means LUT kernel instead of bf16 at batch size 1matvec kernel on L40S · 1x8192 by 8192x8192 matvectime: 175.6 us -> 49.5 us (3.7x)Improved
1-bit k-means LUT kernel instead of bf16matvec kernel on L40S · 1x8192 by 8192x8192 matvectime: 175.6 us -> 24.0 us (7.6x)Improved
1-bit 31B model instead of 4-bit 12B in generation throughput12B 4-bit k-means · token generation on L40Stokens/s: 79.3 -> 60.7 (12B 4.25-bit vs 31B 1.25-bit)Hurt
increase batch size for quantized kernels (ablation)weight-quantized inference · kernel speedupspeedup: no speedup observed by batch size 256Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy