Research map / Large language models / Reasoning and chain of thought

AsyncSpade: Efficient Test-Time Scaling with Asynchronous Sparse Decoding

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AsyncSpade, a training-free asynchronous, disaggregated sparse decoding system that predicts the next query from a sliding window of recent queries via softmax-normalized ridge regression, so token-level KV selection overlaps with forward inference for test-time scaling.

Problem

Query-aware sparse decoding for long chain-of-thought test-time scaling is bottlenecked by sequential dependence (KV selection must wait for the current query) and coarse page-level selection, which limit serving efficiency (TPOT) and accuracy under high concurrency.

Approach

Predict the next-step query state from a short window of recent queries using softmax-normalized ridge regression with multi-window average pooling, and run token-level KV selection on a separate Cache Rank GPU asynchronously so it overlaps with the Inference Rank's forward pass.

Key results

• AsyncSpade delivers over 20% TPOT reduction compared to Quest and at least 50% reduction compared to full attention on Qwen3-8B and Qwen3-32B.
• AsyncSpade matches or surpasses full attention and sparse baselines in Pass@1 on AIME-24/25, GPQA-Diamond, and MATH-500 with 2k selected tokens.
• Cache management latency is lower than inference time across configurations; required minimal bandwidth (105-229 GB/s) falls within A100/H100 node specs.

Limitations

• Requires dedicated Cache Rank GPU and inter-GPU bandwidth to fully overlap communication
• Evaluated on a limited set of models and reasoning benchmarks
• Described as preliminary work

Experimental setup

Datasets: AIME24 AIME25 GPQA-Diamond MATH500

Benchmarks: AIME24 AIME25 GPQA-Diamond MATH500

Metrics: Pass@1 TPOT decoding FLOPs overlap ratio minimal bandwidth

Baselines: Quest TOVA full attention

Builds on: Quest TOVA ShadowKV FlashInfer

Closest papers by meaning

• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction (ICLR 2026 · similarity 0.91)
• Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding (NeurIPS 2025 · similarity 0.89)
• Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling (ICLR 2026 · similarity 0.88)
• Aligning Tree-Search Policies with Fixed Token Budgets in Test-Time Scaling of LLMs (ICML 2026 · similarity 0.87)
• Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding (NeurIPS 2025 · similarity 0.87)
• SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration (ICLR 2025 · similarity 0.87)