AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys

Xu Yang, Jiapeng Zhang, Dongyang Zhao, Guo Chen, Zhuo Tang

AAAI 2026 · Architectures and efficiency › Attention and transformers · approach: KV cache eviction and compression · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add sign bit to the 2-bit key quantization (ablation)Self-Indexing KVCache · LongBench RB-P (Llama3.1-8B)score: 50.9 -> 54.4Improved
add magnitude-based VQ to sign-only retrieval (ablation)Self-Indexing KVCache · LongBench MF-en (Llama3.1-8B)score: 52.8 -> 54.6Improved
add 64 full-precision sink tokens (ablation)Self-Indexing KVCache · LongBench MF-en (Llama3.1-8B)score: 52.6 -> 54.6Improved
add sink tokens (ablation)Self-Indexing KVCache · LongBench HPQA and RB-P (Llama3.1-8B)score: 55.2 -> 55.7 (HPQA)No clear effect
quantize K and V to 2 bits instead of 16 bitsSelf-Indexing KVCache · LongBench avg (Llama3.1-8B)average score: 58.4 -> 58.2No clear effect
replace KMeans (20 iterations) with one-pass sign-based clusteringcodebook construction · 16K input, batch 10time (ms): 2113.9 -> 88.85Improved
retrieve via compressed-domain LUT-GEMV instead of full dot producttop-k retrieval · 16K input, batch 10time (ms): 0.166 -> 0.039Improved
use custom sparse FlashAttention at 7.5% tokens vs full FlashAttention2attention stage · 16K input, batch 10time (ms): 0.776 -> 0.116Improved
sparse FlashAttention kernel vs Page Attentionattention stage · 16K input, batch 10time (ms): 0.101 -> 0.116No clear effect
add compressed-domain sparse attention on top of FlashAttention2prefill · TT2T 32K promptTT2T (s): 6.18 -> 6.51Hurt
use dynamic compressed-key retrieval instead of static SnapKV pruningKV cache sparsification · RULER 32K (Llama3.1-8B)average score: 70.8 -> 89.2Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 2 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy