Research map / Architectures and efficiency / Attention and transformers
TyphoonMLA: A Mixed Naive-Absorb MLA Kernel For Shared Prefix
Ahmet Caner Yüzügüler, Ahmet Çelik, Jiawei Zhuang, Lukas Cavigelli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use naive formulation for shared prefix and absorb for non-shared part | MLA decode kernel · MMLU DeepSeek-v3 Prompt A | token generation rate (kToken/s): 1.01 -> 1.48 | Improved |
| use hybrid kernel with shorter system prompt (Prompt C) | MLA decode kernel · MMLU DeepSeek-v3 Prompt C | token generation rate (kToken/s): 2.33 -> 2.56 | Improved |
| hybrid naive-absorb vs absorb-only on NPU | MLA decode kernel · attention throughput, NPU | speedup: 1.2x to 3x | Improved |
| hybrid naive-absorb vs FlashMLA/FlashInfer on GPU | MLA decode kernel · attention throughput, GPU | speedup: up to 3.24x | Improved |
| longer shared system prompt (Prompt A vs C) | TyphoonMLA · attention time DeepSeek-v3 MMLU | attention time (ms): 99.1 -> 58.1 (A); 26.9 -> 22.0 (C) | Improved |
| fewer attention heads (Kimi K2 vs DeepSeek-v3) (ablation) | TyphoonMLA · attention throughput | speedup | Improved |
| fall back to absorb kernel below batch threshold (about 61-64) (ablation) | TyphoonMLA · small batch sizes | throughput | Improved |
| store shared prefix KV uncompressed | TyphoonMLA · HBM footprint | overhead (%): about 3% | Hurt |
Closest papers by meaning
- Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access (NeurIPS 2025 · similarity 0.86)
- MHA2MLA-VLM: Enabling DeepSeek’s Economical Multi-Head Latent Attention Across Vision-Language Models (AAAI 2026 · similarity 0.85)
- The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry (ICLR 2024 · similarity 0.85)
- Virtual Head Attention (NeurIPS 2026 · similarity 0.85)
- MVA: Linear Attention with High-order Query-Keys Integration and Multi-level Vocabulary Decomposition (ICML 2025 · similarity 0.85)
- Punctuation-aware Hybrid Trainable Sparse Attention for Large Language Models (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
