Research map / Large language models / Reasoning and chain of thought

When RL Meets Adaptive Speculative Training: A Unified Training-Serving System

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Aurora, a unified training-serving system that continuously trains an EAGLE-3-style speculator from live inference traces (accept/reject signals) via asynchronous RL-style updates with hot-swapped weights, enabling day-0 deployment from scratch.

Problem

Speculative decoding deployments decouple offline drafter training from serving, causing high time-to-serve, delayed end-to-end utility feedback, and degradation under domain drift when the target model or traffic changes.

Approach

Treat the draft model as a policy trained online from accept/reject outcomes of live speculative decoding, with a shadow speculator on a separate training GPU and lazy hot-swap synchronization to the SGLang inference server, plus a Discard Sampling loss on rejected branches.

Key results

• On MiniMax M2.1 (FP8, 4xH200), Aurora (scratch) reaches acceptance length 2.62-2.8 and 1.57x per-request speedup at batch size 1, falling to 1.25x at batch size 32.
• Qwen3-Coder-Next (FP8) reaches 1.51x speedup at batch size 1 and 1.23x at batch size 8, with a slowdown (0.93x) at batch size 32.
• In mixed traffic on Qwen3-8B, an untrained speculator reaches acceptance length 3.08, above the static baseline's 2.63.

Surprising findings

• An untrained speculator trained online from scratch in mixed traffic exceeds the performance of the carefully pretrained static speculator (acceptance 3.08 vs 2.63).
• Speculative decoding is slower than the no-speculation baseline for Qwen3-Coder-Next at batch size 32 (0.93x-0.94x).

Limitations

• Gains shrink or vanish at larger batch sizes (speedup below 1 at batch 32 for Qwen3-Coder-Next).
• Experiments use a stream of about 40k prompts and a single target model per setting, with limited evaluation breadth.
• Anonymous submission; no code or data release stated.

Experimental setup

Datasets: GSM8K Spider CodeSearchNet Finance-Alpaca chatbot_instruction_prompts

Benchmarks: online traffic streams (ordered and mixed) over 5 domains holdout code dataset throughput at batch sizes 1-32

Metrics: speculative acceptance length per-request throughput (tokens/s) speedup over no-speculation

Baselines: Frozen Draft (static speculator) no speculation baseline Online Speculative Decoding

Builds on: EAGLE-3 SGLang AReaL Online Speculative Decoding MTP

Closest papers by meaning

• Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing (NeurIPS 2026 · similarity 0.89)
• EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts (NeurIPS 2026 · similarity 0.89)
• Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning (ICLR 2026 · similarity 0.88)
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios (AAAI 2026 · similarity 0.88)
• LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding (ICML 2026 · similarity 0.87)
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting (NeurIPS 2026 · similarity 0.87)