Research map / Large language models / Reasoning and chain of thought

Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Group Tree Optimization (GTO), which trains a draft model for speculative decoding by maximizing a tree-level expected-acceptance-length reward with group-standardized, PPO-clipped updates against a frozen reference draft.

Problem

Speculative decoding drafters are trained on a single greedy draft path, but decoding uses a re-ranked, pruned draft tree, a training-decoding "draft policy misalignment" that limits speedup.

Approach

Define a sampling-free draft tree reward (log-sum-exp smooth max of per-branch expected acceptance lengths under the target), debias it against a frozen reference draft model, standardize advantages within groups of adjacent prefixes, and optimize a PPO-style clipped surrogate along the longest accepted sequence.

Key results

• GTO improves average acceptance length by about 7.4% and speedup by 7.7% over EAGLE-3 at temperature 0 (e.g., LLaMA-3.1 8B average SR 3.73 vs 3.46).
• On LLaMA-3.1 8B at T=0, GTO reaches 3.73x average speedup and 6.52 average acceptance length versus EAGLE-3's 3.46x and 6.07.
• Applied on top of GRIFFIN and HASS drafts, GTO gives roughly 7-8% average SR gains at T=0 on LLaMA-3 8B and LLaMA-2 7B.

Surprising findings

• A continual-training baseline on EAGLE-3 with extra 200 GPU-hours gives almost no gain, suggesting improvements are algorithmic rather than from extra compute.

Limitations

• Increases training-time compute due to grouped tree construction and two-phase training.
• Theoretical guarantee for T=0 is only a weak sufficient condition, not a strict monotonicity result.

Experimental setup

Datasets: ShareGPT OpenThoughts-114k-math

Benchmarks: MT-Bench HumanEval GSM8K

Metrics: speedup ratio acceptance length

Baselines: PLD Lookahead SPS Medusa Hydra EAGLE EAGLE-2 GRIFFIN HASS EAGLE-3

Builds on: EAGLE-3 EAGLE-2 GRPO PPO HASS

Closest papers by meaning

• TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding (NeurIPS 2026 · similarity 0.93)
• Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning (ICLR 2026 · similarity 0.92)
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting (NeurIPS 2026 · similarity 0.92)
• TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees (NeurIPS 2026 · similarity 0.91)
• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.91)
• UniVer: A Unified Perspective for Multi-step and Multi-draft Speculative Decoding (NeurIPS 2026 · similarity 0.91)