Research map / Architectures and efficiency / Quantization
PolarQuant: Leveraging Polar Transformation for Key Cache Quantization and Decoding Acceleration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PolarQuant, which quantizes post-RoPE key cache by converting 2D key sub-vectors to polar coordinates (quantized radius and angle), and uses lookup tables with fused Triton kernels to speed up query-key products.
Problem
Key cache quantization in LLMs suffers from channel-wise outliers (amplified by RoPE), which degrade low-bit quantization and make decoding with quantized keys costly due to dequantization and RoPE recomputation.
Approach
Group key channels into 2D sub-vectors matching RoPE rotation pairs, convert them to polar coordinates, and quantize radius and angle separately with group-wise asymmetric quantization; the query-key inner product then becomes a lookup over the finite set of quantized 2D states, avoiding RoPE recomputation.
Key results
- PolarQuant_33 on Llama-3.1-8B-Instruct LongBench reaches 49.53 average, versus 49.26 for Bf16 at 16 bits, with 3.25 effective bits.
- PolarQuant_44 on DeepSeek-R1-Distill-Llama-8B achieves 62.44 overall vs 62.01 Bf16, outperforming KIVI-4 (59.71).
- Kernel latency up to 2.7x faster than KIVI and throughput up to 3.18x higher than Hugging Face baseline at 32K context.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PolarQuant 3/3 instead of KIVI-2 | Llama-3.1-8B-Instruct key cache · LongBench | average score: 48.68 -> 49.53 | Helped |
| Use PolarQuant 3/3 instead of KIVI-2 | Llama-2-7B-Chat key cache · LongBench | average score: 30.48 -> 31.20 | Helped |
| Use PolarQuant 4/4 instead of KIVI-4 | Qwen-2.5-1.5B-Instruct key cache · LongBench | average score: 36.53 -> 36.41 | Hurt |
| Use PolarQuant 4/4 instead of KIVI-4 | DeepSeek-R1-Distill-Qwen-1.5B key cache · Reasoning benchmarks | overall score: 45.30 -> 47.69 | Helped |
| Use PolarQuant 4/4 instead of ZipCache-4 | DeepSeek-R1-Distill-Llama-8B key cache · Reasoning benchmarks | overall score: 61.13 -> 62.44 | Helped |
| Use PolarQuant 3/3 instead of KIVI-2 with the custom QK kernel | Llama-3.1-8B-Instruct decoding · 32K-context QK multiplication, batch size 8 | latency (s): 2.35 -> 1.03 | Helped |
| Set group size to 32 (ablation) | PolarQuant 4/4 · LongBench, Llama-3.1-8B-Instruct | average score: 49.39 -> 49.50 | No clear effect |
| Set group size to 64 (ablation) | PolarQuant 4/4 · LongBench, Llama-3.1-8B-Instruct | average score: 49.39 -> 49.33 | No clear effect |
| Set group size to 256 (ablation) | PolarQuant 4/4 · LongBench, Llama-3.1-8B-Instruct | average score: 49.39 -> 49.58 | No clear effect |
| Allocate 5 bits to radii and 3 bits to angles (ablation) | PolarQuant, 4.25-bit configuration · LongBench, Llama-3.1-8B-Instruct | average score: 49.39 -> 49.41 | No clear effect |
| Allocate 3 bits to radii and 5 bits to angles (ablation) | PolarQuant, 4.25-bit configuration · LongBench, Llama-3.1-8B-Instruct | average score: 49.39 -> 49.51 | No clear effect |
| Allocate 4 bits to radii and 2 bits to angles (ablation) | PolarQuant, 3.25-bit configuration · LongBench, Llama-3.1-8B-Instruct | average score: 49.53 -> 47.66 | Hurt |
Surprising findings
- PolarQuant slightly exceeds full-precision accuracy on Llama-3.1-8B-Instruct LongBench at 3-bit key quantization.
- Token-wise quantization baselines (Int-4, ZipCache) collapse on Qwen-2.5-1.5B-Instruct, which has extreme key outliers from attention bias.
Limitations
- Evaluated only on RoPE-based decoder-only LLMs; effectiveness on other position encodings or attention variants is untested.
- Newer LLM backbones were not evaluated due to compute and time limits.
- Value cache kept in full precision in main experiments, so the full KV compression story depends on separate value quantization results.
Experimental setup
Benchmarks: LongBench GSM8K AIME24 AIME25 MATH GPQA
Metrics: accuracy F1 score latency throughput memory usage
Baselines: Int-4 Int-3 ZipCache KIVI-4 KIVI-2 QJL SnapKV
Builds on: KIVI KVQuant RoPE ZipCache QJL
Closest papers by meaning
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024 · similarity 0.90)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.88)
- ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification (NeurIPS 2024 · similarity 0.88)
- Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models (ICML 2025 · similarity 0.88)
- CommVQ: Commutative Vector Quantization for KV Cache Compression (ICML 2025 · similarity 0.88)
- Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.