Research map / Architectures and efficiency / Quantization
Ultra-low-bit LLM quantization
Pushes language models to FP4, 1-bit, and multi-bit formats with scaling-law analysis.
56 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2020–2026) · an approach in Quantization, keyword cluster “mixed precision · floating · point”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
12 papers in 2023–24 and 38 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 10.2% to 11.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2020 · Bayesian Bits: Unifying Quantization and Pruning (NeurIPS 2020 · 79 citations)
- 2021 · Square Root Bundle Adjustment for Large-Scale Reconstruction (CVPR 2021 · 29 citations)
- 2022 · Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer (NeurIPS 2022 · 31 citations)
- 2023 · The case for 4-bit precision: k-bit Inference Scaling Laws (ICML 2023 · 23 citations)
- 2024 · JointSQ: Joint Sparsification-Quantization for Distributed Learning (CVPR 2024 · 18 citations)
- 2025 · MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · 4 citations)
- 2026 · Mix-QSAM2: Mixed-Precision Quantization for High Fidelity Segmentation in Resource Constrained Scenarios (AAAI 2026 · 1 citation)
Earliest papers here
- Bayesian Bits: Unifying Quantization and Pruning (NeurIPS 2020 · 79 citations)
- Square Root Bundle Adjustment for Large-Scale Reconstruction (CVPR 2021 · 29 citations)
- Layer-Wise Searching for 1-Bit Detectors (CVPR 2021 · 28 citations)
What has been tried
36 papers here report 455 measured changes (a component added, removed or swapped, compared with a baseline): helped 66%, no clear effect 7%, hurt 25%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add GD-Loss to PTMQ reconstruction | ImageNet, 3-bit and 5-bit · top-1 accuracy | up to 3%; up to 1% | Helped | PTMQ: Post-training Multi-Bit Quantization of Neural Networks (AAAI 2024 · 10 citations) |
| Add auxiliary queries | COCO · AP | 35.2 -> 41.0 | Helped | AQ-DETR: Low-Bit Quantized Detection Transformer with Auxiliary Queries (AAAI 2024 · 8 citations) |
| Add outlier-driven mixed quantization (OMQ) | ImageNet 256×256, W3A4 · IS | 134.30 -> 193.18 | Helped | MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · 4 citations) |
| Align exponent to the maximum rather than the median | Softmax · loss | 9.6x | Hurt | Pushing the Limits of BFP on Narrow Precision LLM Inference (AAAI 2025 · 3 citations) |
| Replace MXFP4 with NVFP4 | CogVideoX layers · CosSim | 98.37% -> 99.52% | Helped | SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training (NeurIPS 2025 · 3 citations) |
| Add GBC to fixed 8-bit quantization | Urban100 ×4 · PSNR | 26.01 -> 25.97 | Hurt | Thinking in Granularity: Dynamic Quantization for Image Super-Resolution by Intriguing Multi-Granularity Clues (AAAI 2025 · 2 citations) |
| remove the Hadamard transform | HellaSWAG · accuracy | 39.18 -> 38.03 | Hurt | QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · 1 citation) |
| Use salience-determined bit allocation instead of ILP allocation | WikiText2, 2-bit · perplexity | 17.55 -> 14.58 | Helped | SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models (ICML 2025 · 1 citation) |
| use linear-block rather than expert-level bitwidth allocation | 5-bit weight-activation quantization · PPL | 6.32 -> 6.11 | Helped | MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design (ICML 2025 · 1 citation) |
| Add activation clamping | LLaMA 1.3B activation tensors, 30,000 iterations · cosine similarity | 92.19% -> 98.83% | Helped | Optimizing Large Language Model Training Using FP4 Quantization (ICML 2025 · 1 citation) |
| Use smaller blocks in quantization-sensitive sublayers (dn, Q, K) | LLaMA3-8B · accuracy | 66.60 -> 69.00 | Helped | BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference (ICML 2025 · 0 citations) |
| smooth both Q and K instead of leaving them unsmoothed | CogvideoX layers · CosSim | 80.04% -> 99.46% | Helped | SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization (ICML 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 20 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- WikiText2 · 6 of 20 papers
- C4 · 5 of 20 papers
- ImageNet · 4 of 20 papers
- HellaSwag · 3 of 20 papers
- GSM8K · 3 of 20 papers
- PIQA · 2 of 20 papers
- COCO · 2 of 20 papers
- LAMBADA · 2 of 20 papers
Baselines
- GPTQ · 4 of 20 papers
- QuaRot · 3 of 20 papers
- AWQ · 2 of 20 papers
- OmniQuant · 2 of 20 papers
- PB-LLM · 2 of 20 papers
- QuIP · 2 of 20 papers
- SageAttention · 2 of 20 papers
- FlashAttention2 · 2 of 20 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 50 papers; from OpenAlex and the papers' affiliations).
- Huawei · 4 papers
- Tsinghua University · 4 papers
- Nanjing University · 4 papers
- Google · 4 papers
- Beihang University · 3 papers
- ETH Zurich · 3 papers
- Institute of Science and Technology Austria · 3 papers
- Swiss Federal Institute of Technology · 3 papers
- Huazhong University of Science and Technology · 2 papers
- Anhui University · 2 papers
Use it
Download BibTeX of 30 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Mixed-precision quantization (92 papers)
- Low-precision training formats (58 papers)
- Binary neural networks (27 papers)
Elsewhere in the topic
- Post-training quantization of LLMs (136 papers)
- Post-training quantization of ViTs (105 papers)
- Post-training weight quantization (100 papers)
- KV cache quantization (72 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).