Research map / Large language models / Reasoning and chain of thought

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Benchmarks nine speculative decoding methods (model-based, training-based, n-gram-based, and hybrid) on Qwen3 and DeepSeek-R1-Distill-Llama 8B under Best-of-N and multi-round thinking for test-time scaling, finding n-gram methods like SAM strong and hybrid SAM[EAGLE-3] best overall.

Problem

Speculative decoding is well studied for generic single-pass generation, but its efficacy for the repetition-rich, long-reasoning workloads of LLM test-time scaling (Best-of-N and multi-round thinking) has not been systematically compared.

Approach

Provides a head-to-head benchmark under consistent test-time scaling protocols, comparing training-free n-gram drafters (especially suffix-automaton SAM) against trained draft models (EAGLE-3, SpS) and a hybrid that switches between SAM and EAGLE-3, and analyzes how speedups evolve across turns and temperatures.

Key results

• Under multi-turn thinking at T=0 on Qwen3-8B, hybrid SAM[EAGLE-3] reaches 3.97x overall speedup, versus 2.91x for EAGLE-3 and 2.28x for SAM.
• SAM speedup on Qwen3-8B drops about 22% (2.28x to 1.78x) when temperature rises from 0 to 0.6, showing sensitivity of n-gram methods to temperature.
• Recycling achieves the highest MAT among n-gram methods (about 2.9-3.0) and is least temperature-sensitive, but its draft overhead limits speedup (about 2.1x).

Surprising findings

• SpS achieves the highest acceptance length (MAT about 7) yet yields speedups below 1x on Qwen3-8B because the 0.6B draft model's cost offsets its gains.
• REST performs poorly (about 1.0x speedup) likely because its external chat datastore overlaps little with long chain-of-thought traces.

Limitations

• Evaluation uses only 120 problems (30 each from AIME24, AIME25, MATH-500, GPQA), a small suite.
• Only 8B-class (plus 4B/14B for Qwen3) models and two test-time scaling frameworks are covered; hybrid strategy is a simple heuristic.
• EAGLE-3 draft models come from third-party teams, so training quality confounds comparisons.

Experimental setup

Datasets: AIME 2024 AIME 2025 MATH-500 GPQA

Benchmarks: AIME 2024 AIME 2025 MATH-500 GPQA

Metrics: Mean Accepted Tokens (MAT) walltime speedup ratio

Baselines: SpS EAGLE-3 PLD REST Lookahead PIA SAM Recycling

Builds on: EAGLE-3 speculative sampling prompt lookup decoding Lookahead decoding Recycling

Closest papers by meaning

• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.94)
• Reward-Guided Speculative Decoding for Efficient LLM Reasoning (ICML 2025 · similarity 0.93)
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction (ICLR 2026 · similarity 0.93)
• RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding (ICML 2025 · similarity 0.92)
• SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration (ICLR 2025 · similarity 0.92)
• A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.92)