AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving

Hui Zeng, Daming Zhao, Pengfei Yang, WenXuan Hou, Tianyang Zheng, Hui Li, Weiye Ji, Jidong Zhai

AAAI 2026 · Architectures and efficiency › Attention and transformers · approach: KV cache eviction and compression · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use layer- and time-adaptive pruning instead of H2OQwen-7B · Math500accuracy (%): 68.00 -> 85.40Improved
use adaptive layer allocation instead of static PyramidKVLlama-8B · Math500accuracy (%): 43.00 -> 77.80Improved
use Lethe instead of StreamingLLMLlama-8B · Math500accuracy (%): 34.40 -> 77.80Improved
prune KV cache with Lethe versus full cacheQwen-32B · Math500accuracy (%): 88.00 -> 80.00Hurt
prune KV cache with Lethe versus full cacheQwen-7B · Math500accuracy (%): 86.40 -> 85.40Hurt
prune KV cache with Lethe at batch size 8Llama-8B · throughputtokens/s: 115.6 -> 217.8Improved
prune KV cache with Lethe at batch size 32Qwen-7B · throughputtokens/s: OOM -> 245.0Improved
prune KV cache with Lethe at batch size 1Qwen-7B · throughputtokens/s: 33.1 -> 34.6No clear effect
prune KV cache with Lethe at batch size 8Llama-8B · generation memoryMB: 65096 -> 5383Improved
prune KV cache with Lethe at batch size 8Qwen-32B · generation memoryMB: 18633 -> 18639No clear effect
increase sparse_ratio up to 400 (ablation)Lethe · Math500 / MMLUaccuracyImproved
lower sparse_ratio below 400 (ablation)Lethe · Math500 / MMLUaccuracyHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy