Research map / Large language models / Reasoning and chain of thought

SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SpecReason, which uses a small reasoning model to draft intermediate thinking steps and a base LRM to score and accept or regenerate them, cutting LRM inference latency while keeping or improving accuracy, and combining with speculative decoding.

Problem

LRM inference with long chain-of-thought is slow due to long autoregressive reasoning sequences, and existing speculative decoding requires exact token-level equivalence, leaving semantic flexibility in thinking tokens unexploited.

Approach

Exploits approximation tolerance of reasoning: a lightweight reasoning model speculates whole reasoning steps, and the base LRM scores each step with a single-prefill utility score (accept if above threshold, else regenerate), operating at semantic rather than token level and complementary to speculative decoding.

Key results

• SpecReason achieves 1.4-3.0x latency speedup over vanilla LRM inference while improving accuracy by 0.4-9.0% across model combinations.
• SpecReason combined with speculative decoding yields an additional 8.8-58.0% latency reduction over speculative decoding alone.
• On QwQ-32B + R1-1.5B, the acceptance threshold trades off latency and accuracy; on AIME, accuracy rises from 22.3% to 39.3% as threshold goes from 3 to 7 while latency grows from 109.4s to 261.9s.

Measured effects reported in the paper

Surprising findings

• Accuracy improves over the base model in many settings because the small model is less verbose, reducing thinking tokens under the 8192 budget.
• Base model's single-token utility scores correlate well with Math-Shepherd PRM scores on AIME.

Limitations

• Static discrete threshold scoring offers coarse control over the accuracy-latency tradeoff.
• Performance gap between small and base models on AIME and GPQA limits acceptance and accuracy gains.
• Sequential inference on colocated models leaves overlap/pipelining unexplored.

Experimental setup

Datasets: AIME MATH500 GPQA Diamond

Benchmarks: AIME MATH500 GPQA Diamond

Metrics: pass@1 end-to-end latency token count

Baselines: vanilla LRM inference speculative decoding (SpecDecode) SpecReason+Decode Sky-T1-Flash Dynasor-CoT

Builds on: speculative decoding vLLM Math-Shepherd

Closest papers by meaning

• Scaling Speculative Decoding with Lookahead Reasoning (NeurIPS 2025 · similarity 0.92)
• SpecExit: Accelerating Large Reasoning Model via Speculative Exit (ICML 2026 · similarity 0.92)
• Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling (ICLR 2026 · similarity 0.90)
• Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference (ICLR 2025 · similarity 0.89)
• SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism (ICLR 2026 · similarity 0.88)
• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.88)