Research map / Large language models / Reasoning and chain of thought

ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ATTS, a training-free asynchronous test-time scaling framework that uses conformal p-values and ordinal classification to control target-model rejection rates in draft/target speculative sampling.

Problem

Test-time scaling with speculative decoding is hampered by target-model memory peaks and synchronization overhead from global ranking/softmax normalization during rejection sampling across parallel and sequential scaling.

Approach

Reinterprets the conformal miscoverage rate as the target model's rejection rate and replaces normalized conformity scores with conformal p-values computed against a calibration set (online calibration from pre-sampled outputs), so each candidate can be accepted or rejected asynchronously without global softmax or ranking.

Key results

• Up to 56.7x speedup in test-time scaling and 4.14x throughput improvement with 16 samples and no turn limit, with no accuracy loss claimed.
• Speedups of up to 22.22x on AIME24 with s1.1-32B target (Qwen2.5-7B draft) under marginal coverage while matching target accuracy in some settings.
• With DeepSeek-R1-Distill-Qwen-1.5B draft and DeepSeek-R1-Distill-Llama-70B target, 128 samples reaches o3-mini (high) performance on AIME 2025 per the authors.

Surprising findings

• Using a non-reasoning draft model yields the highest acceleration, though reasoning drafts give better scaling performance.
• Speedup can fall below 1 on simpler datasets (MATH, AMC23) when the draft's reasoning chains are longer than the target's.

Limitations

• Speedup is below 1 on simpler datasets when draft reasoning chains exceed target chain length.
• Theoretical guarantees are stated with exchangeability assumptions and the paper's proofs contain inconsistencies between marginal and conditional coverage statements.
• Large-scale experiments are limited by compute, so results at 128 samples fall short of GPT-5 performance.

Experimental setup

Datasets: MATH AIME24 AIME25 AMC23 OlympiadBench

Benchmarks: MATH100 AIME24 AIME25 AMC23 OlympiadBench

Metrics: accuracy end-to-end speedup throughput token consumption latency rejection rate control (budget prediction error)

Baselines: SpecReason SpecThink (Speculative Thinking)

Builds on: conformal prediction speculative decoding SpecReason Speculative Thinking SGLang

Closest papers by meaning

• Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling (ICLR 2026 · similarity 0.93)
• AsyncSpade: Efficient Test-Time Scaling with Asynchronous Sparse Decoding (ICML 2026 · similarity 0.91)
• Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.90)
• Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scaling (NeurIPS 2025 · similarity 0.90)
• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.90)
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios (AAAI 2026 · similarity 0.90)