Research map / Architectures and efficiency / Quantization
KV-Cache Transform Coding for Compact Storage in LLM Inference
Konrad Staniszewski, Adrian Łańcucki
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compress KV cache with KVTC at 16x (about 20x after DEFLATE) | Llama 3.1 8B, MN-Minitron 8B, Mistral NeMo 12B · GSM8K, MMLU, Qasper, LITM, RULER-VT | accuracy: GSM8K 56.8 -> 56.9 (Llama 3.1 8B) | No clear effect |
| raise KVTC compression to 64x (ablation) | KVTC on Llama 3.1 8B · LITM | accuracy: 99.3 -> 90.2 | Hurt |
| raise KVTC compression to 64x (ablation) | KVTC on MN-Minitron 8B · LITM | accuracy: 99.3 -> 59.5 | Hurt |
| compress the 4 attention sink tokens (s=0 instead of s=4) (ablation) | KVTC 64x on Llama 3.1 8B · benchmark tasks | accuracy | Hurt |
| compress the most recent 128 tokens (w=0 instead of w=128) (ablation) | KVTC 64x on Llama 3.1 8B · benchmark tasks | accuracy | Hurt |
| use KVTC instead of KIVI 2-bit | Llama 3.1 8B · LITM | accuracy: 88.8 -> 99.3 | Improved |
| use KVTC instead of token eviction (TOVA, H2O) | Llama 3.1 8B · MMLU | accuracy: 44.8 -> 60.1 | Improved |
| use KVTC instead of xKV | Llama 3.1 8B · Qasper | F1: 35.6 -> 40.7 | Improved |
| add DEFLATE entropy coding after quantization (ablation) | KVTC · compression ratio | compression ratio: approximately 1.23x on top of quantization | Improved |
| KVTC at 16x on reasoning model | Qwen 2.5 R1 7B · LiveCodeBench | accuracy: 36.7 -> 31.6 | Hurt |
| KVTC at 8x on reasoning model | Qwen 2.5 R1 1.5B and 7B · LiveCodeBench | accuracy: 16.4 -> 16.1 (1.5B); 36.7 -> 36.5 (7B) | No clear effect |
| KVTC on Llama 3.3 70B with four-GPU pipeline | Llama 3.3 70B Instruct · MATH-500 | accuracy: 75.6 -> 72.6 at 20x | Hurt |
Closest papers by meaning
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.92)
- ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference (NeurIPS 2025 · similarity 0.92)
- Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction (NeurIPS 2026 · similarity 0.92)
- RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression (ICML 2025 · similarity 0.92)
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.91)
- MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
