Research map / Architectures and efficiency / RNNs and normalization
Semantic Cache Distillation: Efficient State Transfer via Reuse and Selective Patching
Qianli Ma, Zhiqing Tang, Hanshuai Cui, Zhi Yao, Weijia Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Patch at transition layers to Reuse-only (ablation) | SCD (Mistral-7B) · CMRC2018 | F1: 0.6421 -> 0.7850 | Improved |
| add Patch at transition layers to Reuse-only (ablation) | SCD (Mistral-7B) · WikiText-2 | perplexity: 6.11 -> 4.65 | Improved |
| add Patch at transition layers to Reuse-only (ablation) | SCD (Mistral-7B) · CMRC2018 | TTFT (ms): 118.2 -> 125.2 | Hurt |
| patch 1 layer instead of none (ablation) | SCD (Mistral-7B) · WikiText-2 | perplexity: 6.788 -> 5.971 | Improved |
| patch top-3 layers instead of none (ablation) | SCD (Mistral-7B) · WikiText-2 | perplexity: 6.788 -> 5.326 | Improved |
| patch top-5 layers instead of none (ablation) | SCD (Mistral-7B) · WikiText-2 | perplexity: 6.788 -> 5.017 | Improved |
| patch top-8 layers instead of none (ablation) | SCD (Mistral-7B) · WikiText-2 | perplexity: 6.788 -> 4.816 | Improved |
| patch 8 layers instead of 1 adds latency (ablation) | SCD (Mistral-7B) · WikiText-2 | TTFT (ms): 115.9 -> 145.9 | Hurt |
| add Patch (Reuse-only vs SCD) latency breakdown recompute phase (ablation) | SCD (Mistral-7B) · WikiText-2 | total TTFT (ms): 112.9 -> 124.1 | Hurt |
| replace DroidSpeak-style selective recompute with SCD | Mistral-7B pair · CMRC2018 | F1: 0.7052 -> 0.7850 | Improved |
| replace DroidSpeak-style selective recompute with SCD | Mistral-7B pair · CMRC2018 | TTFT (ms): 198.7 -> 125.2 | Improved |
| quantize transferred KV to 4-bit instead of BF16 | Raw KV transfer · CMRC2018 | F1: 0.2725 -> 0.1289 | Hurt |
Closest papers by meaning
- Cache-to-Cache: Direct Semantic Communication Between Large Language Models (ICLR 2026 · similarity 0.90)
- ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference (NeurIPS 2025 · similarity 0.89)
- CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences (ICLR 2025 · similarity 0.88)
- MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.88)
- Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference (ICML 2024 · similarity 0.88)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
