Research map / Architectures and efficiency / Attention and transformers
Draft-based Approximate Inference for LLMs
Kevin Galim, Ethan Ewer, Wonjun Kang, Minjae Lee, Hyung Il Koo, Kangwook Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SpecKV draft lookahead instead of SnapKV | KV dropping Cmax=256 · LongBench All, Qwen2.5 | average score: 42.98 -> 46.06 | Improved |
| use SpecKV instead of LAQ++ | KV dropping Cmax=256 · LongBench All, Llama-3 70B | average score: 48.43 -> 48.80 | Improved |
| use SpecKV instead of SnapKV on SingleQA | KV dropping Cmax=256 · LongBench SingleQA, Llama-3 70B | score: 55.88 -> 51.80 | Hurt |
| cascade SpecPC prompt compression to 2048 tokens before SpecKV | SpecKV · LongBench All, Llama-3 70B | average score: 48.80 -> 51.60 | Improved |
| cascade SpecPC before SpecKV | SpecKV · LongBench All, Qwen2.5 | average score: 46.06 -> 46.48 | Improved |
| use SpecPC instead of SpecPrefill | prompt compression Cmax=1024 · LongBench All, Qwen2.5 | average score: 42.70 -> 43.66 | Improved |
| use SpecPC instead of SpecPrefill | prompt compression Cmax=1024 · LongBench All, Llama-3 70B | average score: 48.37 -> 48.44 | No clear effect |
| use SpecPC instead of LLMLingua-2 | prompt compression Cmax=1024 · LongBench All, Qwen2.5 | average score: 30.90 -> 43.66 | Improved |
| KV dropping with SpecKV vs dense target | KV dropping Cmax=256 · LongBench All, Qwen2.5 | average score: 47.78 -> 46.06 | Hurt |
| use larger draft model (ablation) | SpecKV / SpecPC · RULER-32K | downstream score | Improved |
| use draft lookahead tokens (ablation) | SpecKV · RULER | accuracy | Improved |
| increase lookahead tokens (ablation) | SpecPC · RULER | accuracy | No clear effect |
Closest papers by meaning
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.92)
- ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference (NeurIPS 2025 · similarity 0.92)
- QUEST: Query-Aware Sparsity for Efficient Long-Context LLM Inference (ICML 2024 · similarity 0.91)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.91)
- SCBench: A KV Cache-Centric Analysis of Long-Context Methods (ICLR 2025 · similarity 0.91)
- ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference (ICML 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
