Research map / Architectures and efficiency / Attention and transformers
Vortex: Efficient and Programmable Sparse Attention Serving
Zhuoming Chen, Xinrui Zhong, Qilong Feng, Ranajoy Sadhukhan, Yang Zhou, Michael Shieh, Zhihao Jia, Beidi Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents Vortex, a system for efficiently deploying and programming sparse attention algorithms for LLM serving, aimed at easing experimentation by human researchers and AI agents.
Problem
Deploying and evaluating new sparse attention algorithms for LLM serving at scale is engineering-intensive, slowing researchers and AI agents exploring sparse attention designs.
Approach
A system that combines a Python-embedded domain-specific language with a serving backend so that new sparse attention algorithms can be programmed and deployed without heavy engineering effort.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| run block top-k sparse attention on Vortex instead of full attention | Qwen3 dense models · AIME24 | throughput speedup: up to 3.60x at 5 pp accuracy budget | Improved |
| run Quest on Vortex instead of full attention | Qwen3 dense models · AMC23/AIME24 | throughput speedup: 2.73x and 2.98x | Improved |
| use sparse attention at 16K prompt, 8 req/s | Qwen3 · user-side latency | P95 TPOT: 11.7x (block top-k), 12.8x (Quest) lower | Improved |
| agent-driven 18-hour optimization loop | Claude Code on Qwen3-1.7B · AIME24 | throughput and accuracy: 3,437 -> 11,894 tok/s; accuracy 38.54 -> 38.96 | Improved |
| make scoring rope-aware (use full fused-latent dot product incl. rope) (ablation) | MLA block-sparse flow · AIME26 GLM-4.7-Flash | mean@16: 0.60-0.63 -> 0.75 | Improved |
| mask both critical channel groups g3 and g7 in routing (ablation) | Quest/block top-k routing · RULER (Qwen3-4B/8B) | accuracy | Hurt |
| mask only one of g3 or g7 (ablation) | sparse routing · RULER (Qwen3-4B/8B) | accuracy | No clear effect |
| use sparse attention on MiniMax-M2.7 with TP=4 | block top-k · AIME26 MiniMax-M2.7 | mean@16 and speedup: 0.83 -> 0.84; 3,341 -> 4,110 tok/s | Improved |
| add stochastic early termination and remapping to radix top-k (ablation) | radix top-k kernel · top-k kernel latency | speedup at recall>0.97: 1.30-1.62x, average 1.49x | Improved |
| replace dense attention with sparse attention at BS=16, 32k | Qwen3 decode block · per-step decode latency | end-to-end speedup: 3.78-4.81x (block top-k); 3.46-4.33x (Quest) | Improved |
| increase top-k budget beyond ~125 blocks (block size 16) (ablation) | block top-k · accuracy on sweeps | accuracy | No clear effect |
Closest papers by meaning
- Twilight: Adaptive Attention Sparsity with Hierarchical Top-$p$ Pruning (NeurIPS 2025 · similarity 0.87)
- Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix (ICLR 2025 · similarity 0.86)
- SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024 · similarity 0.86)
- Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction (NeurIPS 2025 · similarity 0.86)
- More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding (NeurIPS 2026 · similarity 0.86)
- Low-Rank Approximation for Sparse Attention in Multi-Modal LLMs (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
