Research map / Large language models / Reasoning and chain of thought

SpecExit: Accelerating Large Reasoning Model via Speculative Exit

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SpecExit, which extends the MTP/draft-model hidden states with auxiliary heads predicting confidence, progress, and remaining reasoning length so that speculative decoding can exit reasoning early without probing overhead.

Problem

Large reasoning models overthink, generating unnecessarily long chain-of-thought with high end-to-end latency; existing early-exit methods rely on probing that adds overhead and generalizes poorly across tasks.

Approach

Augment the speculative-decoding draft (MTP) head with extra linear outputs that regress confidence, reasoning progress, and remaining reasoning length from hidden states, trained with multi-task loss and gradient-based dynamic weighting, and use them with EWMA smoothing to trigger a speculative early exit at paragraph delimiters without modifying the target model.

Key results

• SpecExit reduces Qwen3-4B-Thinking-2507 GSM8K tokens from 1414 to 649 with latency 75.8 vs 140.3 for EAGLE3 (about 1.9x speedup).
• On DeepSeek-R1-Distill-Llama-8B GSM8K, SpecExit cuts tokens from 976 (EAGLE3) to 333 and latency from 276.9 to 112.6 (about 2.5x).
• Average accuracy with EWMA smoothing is 86.27 vs 85.09 for vanilla think mode on the Qwen3 ablation table, while tokens drop from 10118 to 8469 on average.

Surprising findings

• DEER increases end-to-end latency over the vanilla baseline on several datasets despite shortening outputs.
• For Qwen3-4B-Thinking-2507, inserting the </think> token in NoThink fails to suppress reasoning and yields longer outputs than Think mode.

Limitations

• Gains are reported mainly against EAGLE3 and DEER on a small set of models; the Qwen3 ablation on signals and smoothing is limited to four benchmarks.
• Early-exit thresholds and EWMA factor are hand-set per signal configuration, with threshold choices directly controlling accuracy-efficiency trade-off.

Experimental setup

Datasets: GSM8K MATH500 AIME HumanEvalPlus GPQA Diamond ARC-Challenge

Benchmarks: GSM8K MATH500 AIME HumanEvalPlus GPQA-Diamond ARC-Challenge

Metrics: accuracy token count end-to-end latency

Baselines: Vanilla (Think) NoThink DEER EAGLE3

Builds on: EAGLE3 multi-token prediction speculative decoding DEER

Closest papers by meaning

• SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning (NeurIPS 2025 · similarity 0.92)
• Scaling Speculative Decoding with Lookahead Reasoning (NeurIPS 2025 · similarity 0.90)
• Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling (ICLR 2026 · similarity 0.90)
• Accelerating Large Language Model Reasoning via Speculative Search (ICML 2025 · similarity 0.89)
• AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders (NeurIPS 2025 · similarity 0.88)
• SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding (ICML 2026 · similarity 0.88)