Research map / Architectures and efficiency / Quantization
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
Amir Zandieh, Majid Daliri, Majid Hadian, Vahab Mirrokni
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| quantize KV cache with TurboQuant at 3.5 bits | Llama-3.1-8B-Instruct · LongBench-V1 | average score: 50.06 -> 50.06 | No clear effect |
| quantize KV cache with TurboQuant at 2.5 bits | Llama-3.1-8B-Instruct · LongBench-V1 | average score: 50.06 -> 49.44 | Hurt |
| TurboQuant 2.5 bit vs KIVI 3 bit | KV cache quantization · LongBench-V1 Llama-3.1-8B | average score: 48.50 -> 49.44 | Improved |
| TurboQuant 2.5 bits on Ministral-7B-Instruct | Ministral-7B-Instruct · LongBench-V1 | average score: 49.89 -> 49.62 | Hurt |
| use TurboQuant instead of Product Quantization / RabitQ for indexing | vector quantization · NN search quantization time (d=1536) | quantization time: 239.75 -> 0.0013 | Improved |
| add 1-bit QJL on MSE-quantizer residual (TurboQuant_prod) instead of MSE-only quantizer (ablation) | TurboQuant · inner-product estimation, DBpedia 1536-d | bias | Improved |
| use TurboQuant_prod instead of TurboQuant_mse at higher bit width (ablation) | TurboQuant · inner-product error | error | Mixed |
| use TurboQuant instead of KIVI/SnapKV/PyramidKV on Needle-in-a-Haystack | KV cache compression · Needle-In-A-Haystack recall | recall | Improved |
Later papers that compared with this method
1 later accepted paper reports TurboQuant as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Block Sphere Vector Quantization (NeurIPS 2026 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks (ICLR 2025 · similarity 0.88)
- Block Sphere Vector Quantization (NeurIPS 2026 · similarity 0.88)
- Residual Quantization with Implicit Neural Codebooks (ICML 2024 · similarity 0.87)
- Finite Scalar Quantization: VQ-VAE Made Simple (ICLR 2024 · similarity 0.87)
- Zero-Shot Quantization via Weight-Space Arithmetic (NeurIPS 2026 · similarity 0.87)
- Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
