AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Vortex: Efficient and Programmable Sparse Attention Serving

Zhuoming Chen, Xinrui Zhong, Qilong Feng, Ranajoy Sadhukhan, Yang Zhou, Michael Shieh, Zhihao Jia, Beidi Chen

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Sparse attention for long-context LLMs

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Presents Vortex, a system for efficiently deploying and programming sparse attention algorithms for LLM serving, aimed at easing experimentation by human researchers and AI agents.

Problem

Deploying and evaluating new sparse attention algorithms for LLM serving at scale is engineering-intensive, slowing researchers and AI agents exploring sparse attention designs.

Approach

A system that combines a Python-embedded domain-specific language with a serving backend so that new sparse attention algorithms can be programmed and deployed without heavy engineering effort.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
run block top-k sparse attention on Vortex instead of full attentionQwen3 dense models · AIME24throughput speedup: up to 3.60x at 5 pp accuracy budgetImproved
run Quest on Vortex instead of full attentionQwen3 dense models · AMC23/AIME24throughput speedup: 2.73x and 2.98xImproved
use sparse attention at 16K prompt, 8 req/sQwen3 · user-side latencyP95 TPOT: 11.7x (block top-k), 12.8x (Quest) lowerImproved
agent-driven 18-hour optimization loopClaude Code on Qwen3-1.7B · AIME24throughput and accuracy: 3,437 -> 11,894 tok/s; accuracy 38.54 -> 38.96Improved
make scoring rope-aware (use full fused-latent dot product incl. rope) (ablation)MLA block-sparse flow · AIME26 GLM-4.7-Flashmean@16: 0.60-0.63 -> 0.75Improved
mask both critical channel groups g3 and g7 in routing (ablation)Quest/block top-k routing · RULER (Qwen3-4B/8B)accuracyHurt
mask only one of g3 or g7 (ablation)sparse routing · RULER (Qwen3-4B/8B)accuracyNo clear effect
use sparse attention on MiniMax-M2.7 with TP=4block top-k · AIME26 MiniMax-M2.7mean@16 and speedup: 0.83 -> 0.84; 3,341 -> 4,110 tok/sImproved
add stochastic early termination and remapping to radix top-k (ablation)radix top-k kernel · top-k kernel latencyspeedup at recall>0.97: 1.30-1.62x, average 1.49xImproved
replace dense attention with sparse attention at BS=16, 32kQwen3 decode block · per-step decode latencyend-to-end speedup: 3.78-4.81x (block top-k); 3.46-4.33x (Quest)Improved
increase top-k budget beyond ~125 blocks (block size 16) (ablation)block top-k · accuracy on sweepsaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy