Research map / Large language models / Reasoning and chain of thought
Re-SpS: A Reinforcement Learning Approach to Speculative Sampling
Chenan Wang, Daniel H. Shi, Haipeng Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Re-SpS, a reinforcement learning approach that dynamically adjusts speculative sampling tree-structure hyperparameters (instead of static values used by methods like EAGLE-3) to reduce LLM inference latency.
Problem
Inference latency of LLMs; state-of-the-art tree-based speculative sampling (e.g., EAGLE-3) uses static tree-structure hyperparameters that limit flexibility and efficiency across diverse contexts and decoding settings.
Approach
Replace static tree-structure hyperparameters of speculative sampling with a learned RL policy that adapts them per context, so the drafting tree adjusts to each situation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SentenceBERT text embedding with reused draft-model hidden-state features as RL state (ablation) | Re-SpS (standard PPO) · Avg speedup vs EAGLE-3, LLaMA 3.1-8B | speedup: 1.044x -> 1.049x | Improved |
| use feature vector instead of text embedding (ablation) | Re-SpS (standard PPO) · Avg speedup vs EAGLE-3, Vicuna-13B | speedup: 1.006x -> 1.028x | Improved |
| increase action cache interval from 1 to 50 steps (ablation) | Re-SpS · LLaMA 3.1-8B (80 questions) | latency and speed | Improved |
| max-entropy PPO instead of standard PPO (ablation) | Re-SpS · LLaMA 3.1-8B | speedup vs EAGLE-3: 1.049x -> 1.025x | Hurt |
| max-entropy PPO instead of standard PPO (ablation) | Re-SpS · Vicuna-13B | speedup vs EAGLE-3: 1.028x -> 1.033x | Improved |
| replace static draft-tree hyperparameters with RL policy | EAGLE-3 · LLaMA 3.3-70B mean over 5 benchmarks | speedup: 4.46× -> 4.88× | Improved |
| replace static draft-tree hyperparameters with RL policy | EAGLE-3 · LLaMA 3.1-8B HumanEval | speedup: 3.65× -> 3.89× | Improved |
| replace static hyperparameters with RL policy | EAGLE-3 · CNN/DM (LLaMA 3.1-8B) | speedup: 2.96× -> 2.87× | Hurt |
| replace static hyperparameters with RL policy | EAGLE-3 · MT-Bench (LLaMA 3.1-8B) | speedup: 3.39× -> 3.43× | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- EAGLE-3 — EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test (NeurIPS 2025) · better on 8 of 9 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Baselines: EAGLE-3
Builds on: EAGLE-3 speculative sampling
Closest papers by meaning
- Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning (ICLR 2026 · similarity 0.90)
- Accelerated Speculative Sampling Based on Tree Monte Carlo (ICML 2024 · similarity 0.89)
- EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts (NeurIPS 2026 · similarity 0.88)
- Dynamic Speculative Agent Planning (ICLR 2026 · similarity 0.88)
- Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening (ICML 2026 · similarity 0.87)
- Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
