Research map / Architectures and efficiency / Attention and transformers
QuoKA: Query-Oriented KV Selection for Efficient LLM Prefill
Dalton Jones, Junyoung Park, Matthew J Morse, Mingu Lee, Matthew Harper Langston, Christopher Lott
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use QuoKA instead of SampleAttention for KV selection in chunked prefill | Llama-3.2-3B-Instruct · RULER 32k, B_SA=1024 | score: 31.73 -> 57.01 | Improved |
| use QuoKA instead of SampleAttention | Qwen-3-4B · RULER 32k, B_SA=1024 | score: 40.72 -> 74.83 | Improved |
| use QuoKA instead of SampleAttention on GPT-OSS-20B at 4k | GPT-OSS-20B · RULER 4k, B_SA=1024 | score: 76.20 -> 78.92 | No clear effect |
| use QuoKA instead of SampleAttention for LongBench with small budget | Llama3.2-3B-Instruct · LongBench, B_SA=512 | normalized accuracy: 0.738 -> 0.945 | Improved |
| use QuoKA with 25% KV budget instead of full attention | Qwen2.5-3B · RULER 32768 | score: 71.69 -> 68.84 | Hurt |
| use QuoKA with 25% KV budget instead of full attention | Qwen3-4B · RULER 32768 | score: 88.54 -> 87.87 | No clear effect |
| score keys by cosine similarity instead of dot product (ablation) | QuoKA · RULER | subselection quality: more than 10% | Improved |
| reduce query subselection to N_Q = B_CP/16 (ablation) | QuoKA · LongBench and RULER | accuracy: about 3% drop | Hurt |
Later papers that compared with this method
1 later accepted paper reports QUOKA as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection (NeurIPS 2026 · better on 5 of 5 shared results)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Loki — LoKi: Low-dimensional KAN for Efficient Fine-tuning Image Models (CVPR 2025) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection (NeurIPS 2026 · similarity 0.90)
- Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction (NeurIPS 2025 · similarity 0.88)
- SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference (NeurIPS 2026 · similarity 0.88)
- Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (NeurIPS 2024 · similarity 0.88)
- Loki: Low-rank Keys for Efficient Sparse Attention (NeurIPS 2024 · similarity 0.88)
- A Training-Free Sub-quadratic Cost Transformer Model Serving Framework with Hierarchically Pruned Attention (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
