Research map / Architectures and efficiency / Quantization

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: KV cache quantization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that KV cache quantization can silently erode refusal alignment in instruction-tuned LLMs and proposes Per-Channel Reduction (PCR), a 20-prompt diagnostic that predicts which training-free mitigation (FP16 layer protection vs Group-64) will restore safety.

Problem

KV cache quantization is evaluated almost only on perplexity and accuracy, so its effect on safety alignment (refusal behavior) is unknown; the vulnerability varies sharply across models and bit-widths.

Approach

Attributes alignment collapse under KV quantization to channel geometry: safety-critical channels either coincide with activation outliers (outlier-as-safety) or not (outlier-crushes-safety), and layer spread (multi-layer dilution) determines whether per-layer or per-channel fixes work; PCR, the relative drop in flip rate from per-tensor to per-channel quantization at the critical layer, classifie

Key results

• Mistral-7B loses 15.2% of refusals at 4-bit KV quantization while perplexity rises only 1.03x.
• PCR predicts the correct mitigation direction on all nine primary models plus one held-out model (OLMo-2), with 100% directional accuracy on 200 unseen AdvBench prompts.
• The training-free protocol recovers up to 97% of lost alignment (Qwen-2.5-7B protecting L0-1 at FP16: 97.8% recovery) at roughly 35 GPU-minutes per model.

Surprising findings

• Perplexity stays near baseline for Mistral-7B while refusal collapses, so standard metrics miss the failure.
• Attention-based layer selection is never the best mitigation strategy tested.

Limitations

• Validated on a limited set of models; some models (Mixtral, Yi-34B) need 50 calibration prompts due to low per-layer flip rates.
• Evaluation uses simulated quantization for most experiments; real-dtype validation is limited.
• Recovery is zero for models with already-low baseline vulnerability (LLaMA-3.1-8B), and the mechanistic taxonomy is qualitative for multi-layer dilution cases.

Experimental setup

Datasets: AdvBench HarmBench XSTest IFEval custom alignment suite (63 prompts)

Benchmarks: AdvBench HarmBench XSTest IFEval custom alignment suite

Metrics: ConditionalFlip PCR perplexity recovery rate false refusal rate

Baselines: FP16 layer protection attention-based layer selection Group-64 quantization KIVI per-token asymmetric quantization

Builds on: KIVI WildGuard AdvBench HarmBench XSTest IFEval

Closest papers by meaning

• KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference (ICML 2025 · similarity 0.90)
• KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.90)
• KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024 · similarity 0.90)
• MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.89)
• KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache (ICML 2024 · similarity 0.89)
• SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.89)