Research map / Large language models / Reasoning and chain of thought

SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SpecBranch, a training-free speculative decoding framework that runs parallel speculative branches during target verification and uses a hybrid H-RAD predictor (draft confidence plus target features) to set draft length, reporting 1.8–4.5x speedups over autoregressive decoding.

Problem

Standard speculative decoding serializes draft and target model execution, creating pipeline bubbles; parallel variants such as PEARL suffer global invalidation (rollback) when draft tokens are rejected, especially for poorly aligned draft/target pairs.

Approach

Borrowing from branch prediction, the draft model spawns adaptively sized parallel branches at uncertain branch points while the target verifies the previous round; a lightweight MLP over multi-layer target features plus draft confidence (three-way signal) predicts draft length to reduce rollback, and branch speculative sampling keeps the target output distribution.

Key results

• SpecBranch reaches 2.01x average speedup on LLaMA 68M&7B and 3.23x on DeepSeek 1.3B&33B on HumanEval/GSM8K/CNN/DM, versus 1.74x and 2.93x for PEARL.
• On Spec-Bench, SpecBranch achieves 1.8x–4.5x speedups; on Vicuna 68M&13B it averages 1.96x vs PEARL 1.77x.
• Rollback rate is reduced from 66–90% to under 40% for poorly aligned pairs, per the paper's claim.

Surprising findings

• H-RAD (hybrid predictor) contributes more on misaligned pairs while branch resampling contributes more on better-aligned pairs.

Limitations

• Results are reported under greedy target decoding for most experiments; lossless claim for sampling is supported by only a small GSM8K accuracy check.
• Lookahead baseline could not run on LLaMA-3.1 due to transformers version conflict, leaving some comparisons missing.
• Energy and memory overhead remain higher than autoregressive decoding, as the paper acknowledges.

Experimental setup

Datasets: HumanEval GSM8K CNN/DM Spec-Bench

Benchmarks: HumanEval GSM8K CNN/DM Spec-Bench

Metrics: mean accepted length speedup ratio tokens/s rollback rate accuracy

Baselines: SpS AdaEDL Lookahead Decoding PEARL vanilla autoregressive decoding

Builds on: speculative sampling PEARL AdaEAGLE AdaEDL Lookahead decoding

Closest papers by meaning

• PEARL: Parallel Speculative Decoding with Adaptive Draft Length (ICLR 2025 · similarity 0.93)
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting (NeurIPS 2026 · similarity 0.92)
• Speculative Speculative Decoding (ICLR 2026 · similarity 0.92)
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios (AAAI 2026 · similarity 0.92)
• SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding (ICML 2026 · similarity 0.92)
• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.92)