Research map / Architectures and efficiency / Quantization

KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: KV cache quantization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes KV-COBRA, a calibration-time allocator that jointly chooses per-head rank and bit-width (inner C1) and redistributes budget across heads (outer C2), plus an attention-KL reordering, for KV-cache compression with SVD and Hadamard-rotated scalar quantization.

Problem

Per-token KV-cache compressors use uniform rank and bit-width across all attention heads, which is suboptimal at extreme low bit-rates (0.5-2 bpd).

Approach

Treat KV compression as a rate-distortion allocation: per-head joint rank/bit optimization (C1) plus cross-head marginal-distortion equalization (C2), with the SVD basis reordered by an attention-KL weight (query variance times key variance) so the allocation is query-aware; the compression kernel itself is standard.

Key results

• At 0.5 bpd on LLaMA-3.1-8B, KV-COBRA (KL) reaches LongBench F1 of 6.75 vs 6.83 for MSE variant and PPL 31.17 (MSE) vs 37.31 (KL), with the KL variant winning LongBench at 1.0 bpd (11.36 vs 9.25 on LLaMA).
• Reports reduction of cross-head distortion std by 2-8x via C2 allocation relative to uniform per-head budgets.
• Attention primitive is 1.79-1.89x faster than FP16-SDPA at r=16 across 32k-128k context lengths.

Surprising findings

• Replacing the proportional C2 solver with an exact DP reduces the surrogate distortion sum by ~12-16% yet degrades PPL by ~0.5 on average, traced to the Bennett distortion model's mismatch at low bits.
• V-side compression is structurally limited: V spectra are flatter, so C1 pins rank at the ceiling and the allocator adds little; V_bpd=2 collapses on LLaMA and Mistral (PPL >150).

Limitations

• Comparison limited to per-token compressors on the same rank/bit axes; eviction and entropy coding not evaluated jointly.
• Calibration is computed once from 32 Wikitext-2 samples and refresh during generation is not studied.
• Distortion model relies on Bennett's high-resolution approximation, which underestimates distortion at 2 bits by ~1.7x.

Experimental setup

Datasets: Wikitext-2 PTB C4 ARC-c HellaSwag PIQA WinoGrande MMLU LongBench

Benchmarks: Wikitext-2 PTB C4 ARC-c HellaSwag PIQA WinoGrande MMLU LongBench

Metrics: perplexity accuracy F1 wall-clock latency

Baselines: KIVI KVQuant GEAR TurboQuant SVDq KQ-SVD

Builds on: KQ-SVD SVDq QuaRot KIVI Bennett high-resolution quantization theory

Closest papers by meaning

• ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification (NeurIPS 2024 · similarity 0.90)
• CompilerKV: Risk-Adaptive KV Cache Compression via Offline Experience Compilation (NeurIPS 2026 · similarity 0.89)
• EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction (NeurIPS 2026 · similarity 0.89)
• Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries (ICML 2025 · similarity 0.88)
• SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference (NeurIPS 2025 · similarity 0.88)
• SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.88)