Research map / Large language models / Reasoning and chain of thought

Guided Speculative Inference for Efficient Test-Time Alignment of LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Guided Speculative Inference (GSI), which uses draft-model samples with reward-likelihood tilted soft best-of-n and a rejection threshold to approximate the tilted optimal policy of the target model with lower latency.

Problem

Reward-guided decoding from a large target LLM is expensive because soft best-of-n requires autoregressively generating n responses from the target; existing reward-guided speculative decoding (RSD) lacks distributional guarantees.

Approach

Rewrite the tilted optimal policy of the base model as a draft-model distribution tilted by tilted rewards r + (1/beta) log(pi_B/pi_S), run soft best-of-n over draft samples with these tilted rewards, and fall back to target-model soft best-of-n when the selected tilted reward is below a threshold u.

Key results

• On Qwen2.5-Math with n=4, GSI averages 59.5% accuracy versus 57.8% for RSD and 55.5% for S-BoN with the small model, approaching S-BoN with the base model at 61.3%.
• On Qwen3 with n=16, GSI gains 51% more steps per second than S-BoN with the base model at about 3% relative performance cost, and reduces end-to-end latency by up to 28%.
• GSI is reported to beat SPECS by 11.5% on MATH500 at n=4, though SPECS is 1.6% better on OlympiadBench at n=4 (numbers taken from SPECS paper on N=100 subsets).

Surprising findings

• GSI without the rejection step sometimes matches or surpasses S-BoN with the base model, which the authors note empirically verifies their theorem.
• Speculative decoding at large batch sizes is reported to scale poorly, motivating the step-level draft-then-resample design over token-level speculative decoding.

Limitations

• Latency gains are limited by the PRM, which consumes a significant share of runtime; the authors note a smaller PRM would help.
• Comparison to SPECS relies on numbers reported in its paper on different subset sizes, so accuracies are not directly comparable.
• Threshold u and beta are selected empirically and may need tuning, and the paper uses a fixed threshold rather than an n-dependent schedule.

Experimental setup

Datasets: MATH500 OlympiadBench Minerva Math MMLU-STEM GSM8K

Benchmarks: MATH500 OlympiadBench Minerva Math MMLU-STEM GSM8K

Metrics: accuracy inference time per step steps per second acceptance rate

Baselines: RSD S-BoN with draft model S-BoN with base model SPECS

Builds on: reward-guided speculative decoding (RSD) soft best-of-n sampling speculative decoding

Closest papers by meaning

• Reward-Guided Speculative Decoding for Efficient LLM Reasoning (ICML 2025 · similarity 0.93)
• Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling (ICLR 2026 · similarity 0.90)
• Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing (NeurIPS 2026 · similarity 0.90)
• Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment (ICLR 2025 · similarity 0.90)
• A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.90)
• Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding (ICLR 2026 · similarity 0.89)