AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding

Haocheng Xia, Mihir Pamnani, Hanxi Fang, Supawit Chockchowwat, Yongjoo Park

ICML 2026 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
defer RoPE into attention kernel for position-agnostic KV reuseBlock-Attention (vLLM) · skewed document traffictime-to-first-token: 1.37x lower TTFTImproved
defer RoPE into attention kernel for position-agnostic KV reuseBlock-Attention (vLLM) · skewed document trafficinference throughput: 1.40x higherImproved
store single position-agnostic KV copy per documentBlock-Attention (vLLM) · 1 GB cache, high skewVRAM cache hit ratio: 7.27 -> 13.57Improved
store single position-agnostic KV copy per documentBlock-Attention (vLLM) · no-limit cache, mid skewVRAM cache hit ratio: 27.38 -> 29.09Improved
single relative rotation of K inside fused kernel instead of two rotations (ablation)naive deferred encoding · prefilling kernelFLOPs overhead: 3/(2M) -> 3/(4M); 0.59% at M=128Improved
rotate Q only when document offset changes in decoding (ablation)LazyAttention decoding kernel · decodingper-token overhead: 0.13% overheadImproved
use LazyAttention instead of Block-AttentionTulu3-Block-FT · QA average exact matchaccuracy: 71.2 -> 69.2No clear effect
use LazyAttention instead of CacheBlendTulu3-Block-FT · QA average exact matchaccuracy: 67.5 -> 69.2Improved
use deferred rotation under uniform trafficLazyAttention · Uniform document samplingTTFTNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy