Research map / Architectures and efficiency / Quantization
KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
Fei Li, Song Liu, Weiguo Wu, Shiqiang Nie, Jinyu Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| allocate high bits by gradient-based layer importance instead of randomly choosing layers (ablation) | KVmix-k2.19v2.38 · LongBench (avg of 4 models) | accuracy loss vs FP16: 4.06% -> 1.67% | Improved |
| add mixed-precision (k2.19v2.38) over uniform 2-bit quantization (ablation) | KVmix · LongBench (avg of 4 models) | accuracy loss vs FP16: 4.53% -> 1.67% | Improved |
| add recent pivotal context (RPC) full-precision KV retention (ablation) | KVmix-k2.19v2.38 · LongBench (avg of 4 models) | average accuracy: 3.28% decrease without RPC | Improved |
| use importance-aware mixed precision instead of random layer selection (ablation) | KVmix-k2.19v2.38 · GSM8K (Llama-2-7B) | accuracy: 11.97 -> 13.25 | Improved |
| use KVmix instead of uniform 2-bit per-token quantization | Llama-2-7B KV cache · GSM8K / Wikitext-2 | accuracy / perplexity: 0.83 -> 13.25 GSM8K; 11089 -> 8.71 ppl | Improved |
| increase high-bit layer proportion from 20% to 30% (k2.28v2.56) (ablation) | KVmix · LongBench (Llama-2-7B) | average score: 33.714 -> 33.814 | Improved |
| raise RPC proportion above 20% (ablation) | KVmix · accuracy | accuracy | No clear effect |
| use more than 20-30 profiling prompts (ablation) | KVmix profiler · importance analysis | layer importance reliability | No clear effect |
| replace KIVI-2bit-r64 with KVmix-k2.19v2.38 | Llama-2-7B KV cache · LongBench | average accuracy loss: -1.50% accuracy loss | Improved |
| use KVmix instead of FP16 KV cache | Llama-2-7B · inference efficiency | memory reduction / throughput: 4.9x memory reduction; 5.32x throughput | Improved |
| use KVmix instead of KVQuant-3bit-1% | Llama-2-7B · LongBench | accuracy: 33.703 -> 33.714 | No clear effect |
Closest papers by meaning
- KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference (ICML 2025 · similarity 0.94)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.92)
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache (ICML 2024 · similarity 0.92)
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.91)
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024 · similarity 0.91)
- MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
