Research map / Architectures and efficiency / Quantization

ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ScaleBITS, a mixed-precision post-training weight quantization framework for LLMs that uses dynamic sensitivity measured at a quantized reference point, bi-directional channel reordering, and hardware-aligned block partitions with a scalable batched greedy bit allocation search.

Problem

Pushing post-training weight-only quantization of LLMs below 4 bits degrades quality, and existing mixed-precision methods either use hardware-unfriendly fine-grained partitions or coarse/heuristic precision allocation that cannot scale to billions of weights.

Approach

Sensitivity is estimated as a first-order term around the current quantized model (not full precision), weight channels are reordered in both row and column directions so sensitive weights cluster into block-aligned regions, and a batched approximation to greedy knapsack bit allocation assigns precision per 64x128 block under a global budget.

Key results

• At ~2.1 bits on Llama3-70B, ScaleBITS+RTN raises MMLU from 23.02% (uniform RTN-g128) to 58.82% and zero-shot from 35.61% to 64.72%.
• On Llama2-7B at 2.1 bits, ScaleBITS+RTN reaches WikiText-2 perplexity 7.49 vs 15.38 for SlimLLM-g128 and 8.83 for GuidedQuant (2.0 bits).
• Search takes 0.5-1 hour on a single H100 for Llama3-8B, a 3-6x speedup over SlimLLM, with 16-36 search iterations versus about 3e6 for classic greedy.

Surprising findings

• Sensitivity measured at the full-precision model fails to rank layer sensitivities under aggressive quantization, while gradients at the quantized model preserve the ordering.
• Restricting the minimum precision to 0 or 1 bit has minimal impact on the trade-off curve, while capping maximum precision at 4 bits consistently degrades results.

Limitations

• Evaluated only with scalar quantizers (RTN with group size 128); vector quantization and non-uniform quantizers are not combined in experiments.
• Relies on first-order sensitivity approximations and heuristic surrogates for increasing/decreasing precision; the submodularity and monotonicity assumptions are only empirically checked and not used for correctness.
• Kernel is a proof-of-concept Triton implementation, not a fully optimized kernel; channel reordering excludes QK projections from local reordering.

Experimental setup

Datasets: RedPajama WikiText-2 C4

Benchmarks: WikiText-2 WinoGrande PiQA HellaSwag ARC-easy ARC-challenge BoolQ MMLU GSM8K MBPP C4

Metrics: perplexity zero-shot accuracy 5-shot accuracy latency quantization time

Baselines: RTN GPTQ GuidedQuant SlimLLM SlimLLM+ AMQ PB-LLM SqueezeLLM ICQuant

Builds on: GPTQ SlimLLM SqueezeLLM GuidedQuant classic greedy search for submodular knapsack

Closest papers by meaning

• Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs (ICML 2024 · similarity 0.90)
• Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.90)
• LittleBit: Ultra Low-Bit Quantization via Latent Factorization (NeurIPS 2025 · similarity 0.90)
• DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment (NeurIPS 2025 · similarity 0.90)
• QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · similarity 0.90)
• Progressive Mixed-Precision Decoding for Efficient LLM Inference (ICLR 2025 · similarity 0.89)