Research map / Architectures and efficiency / Attention and transformers
Stochastic Sparse Attention for Memory-Bound Inference
Kyle Lee, Corentin Delacour, Kevin Callahan-Coray, Kyle Jiang, Can Yaras, Samet Oymak, Tathagata Srimani, Kerem Yunus Camsari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add stratified sampling instead of iid sampling (ablation) | SANTA (S=16) · GSM8K | accuracy (%): 5.51 -> 39.12 | Improved |
| use systematic instead of independent stratified sampling (ablation) | S2ANTA · GSM8K (S=16) | accuracy (%): 39.12 -> 44.63 | Improved |
| increase sample budget S from 2 to 256 (ablation) | S2ANTA-sys · GSM8K | accuracy (%): 1.11 -> 77.56 | Improved |
| replace iid sampling with systematic sampling at S=32 (ablation) | SANTA · MMLU | accuracy (%): 33.81 -> 45.48 | Improved |
| use speculative per-tile flash allocation with S=256 vs S=2048 (ablation) | S2ANTA-flash · RULER FWE (32k) | accuracy (%): 66.20 -> 94.13 | Improved |
| replace S2ANTA-flash with S2ANTA-prop proportional allocation | S2ANTA kernel · RULER (32k) | smallest S matching SDPA: S=2048 -> S=128 | Improved |
| swap FlashAttention-2 decode for S2ANTA-flash | Llama-3.1-8B-Instruct · end-to-end decode | decode speedup: 1.000x -> 1.250x | Improved |
| use Bernoulli sampling with B=4 instead of B=1 (ablation) | qK^T estimator · GSM8K (BitNet 2B) | accuracy (%): 1.9 -> 64.5 | Improved |
| use mean group query instead of per-head query (ablation) | Bernoulli qK^T (B=4) · GSM8K (BitNet 2B) | K access (%) and accuracy: 97.9 -> 84.7 group access; 64.5 -> 63.8 accuracy | Mixed |
| increase SANTA budget S from 8 to 64 with Bernoulli B=4 (ablation) | Bernoulli qK^T + SANTA · GSM8K (BitNet 2B) | accuracy (%): 55.0 -> 63.8 | Improved |
| increase Bernoulli samples from B=4 to B=8 (ablation) | Bernoulli qK^T · RULER FWE (4k) | accuracy (%): 70.4 -> 89.4 | Improved |
| replace top-k with S2ANTA-sys at matched budget 16 | attention value stage · GSM8K | accuracy (%): 62.88 -> 44.63 | Hurt |
Closest papers by meaning
- More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding (NeurIPS 2026 · similarity 0.89)
- RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression (ICML 2025 · similarity 0.88)
- Efficient Low Rank Attention for Long-Context Inference in Large Language Models (NeurIPS 2025 · similarity 0.88)
- SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference (NeurIPS 2026 · similarity 0.88)
- Value-Guided KV Compression for LLMs via Approximated CUR Decomposition (NeurIPS 2025 · similarity 0.88)
- MUSTAFAR: Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
