Research map / Architectures and efficiency / Quantization
PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs
Tengxuan Liu, Shiyao Li, Jiayi Yang, Tianchen Zhao, Feng Zhou, Xiaohui Song, Guohao Dai, Shengen Yan, Huazhong Yang, Yu Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use equivalent right shift instead of direct right shift when lowering bit-width (ablation) | PM-KVQ · AIME-2024 (DeepSeek-LLaMA-8B, 4-bit) | pass@1: 12.08 -> 38.33 | Improved |
| use equivalent right shift instead of modified right shift (ablation) | PM-KVQ · AIME-2024 (DeepSeek-LLaMA-8B, 4-bit) | pass@1: 28.75 -> 38.33 | Improved |
| add positional interpolation (s=4) to 2,048-token calibration (ablation) | PM-KVQ · AIME-2024-I (DeepSeek-LLaMA-8B) | pass@1: 46.67 -> 48.33 | Improved |
| increase position scaling factor from 4 to 16 (ablation) | PM-KVQ · AIME-2024-I (DeepSeek-LLaMA-8B) | pass@1: 48.33 -> 46.67 | Hurt |
| add progressive quantization with uniform Fbit versus KIVI 2-bit | PM-KVQ · AIME-2024 (DeepSeek-Qwen-7B) | pass@1: 32.08 -> 40.00 | Improved |
| add block-wise memory allocation (BS=32 mixed 2/4) over uniform 2-bit (ablation) | PM-KVQ · AIME-2024 (DeepSeek-Qwen-7B) | pass@1: 40.00 -> 40.21 | No clear effect |
| add block-wise memory allocation (BS=12) over uniform 2-bit (BS=16) (ablation) | PM-KVQ · AIME-2024 (DeepSeek-Qwen-14B) | pass@1: 63.33 -> 67.71 | Improved |
| add block-wise memory allocation (BS=12) over uniform 2-bit (BS=16) (ablation) | PM-KVQ · AIME-2024 (QwQ-32B) | pass@1: 67.29 -> 66.46 | Hurt |
| replace KIVI 2-bit with PM-KVQ on 70B model | DeepSeek-R1-Distill-LLaMA-70B · AIME-2024 | pass@1: 51.88 -> 64.79 | Improved |
| replace KIVI with PM-KVQ on 14B model | DeepSeek-R1-Distill-Qwen-14B · AIME-2024 | pass@1: 48.13 -> 67.71 | Improved |
| replace KVTuner with PM-KVQ | DeepSeek-R1-Distill-Qwen-7B · AIME-2024 | pass@1: 34.17 -> 40.21 | Improved |
| replace KIVI with PM-KVQ | PM-KVQ vs KIVI throughput · Throughput 32K, Qwen-7B | tokens/s: 284.10 -> 269.51 | Hurt |
Closest papers by meaning
- KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference (ICML 2025 · similarity 0.94)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.92)
- MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.92)
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache (ICML 2024 · similarity 0.91)
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024 · similarity 0.91)
- NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
