AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

RepSpec: Structural Re-parameterized Draft Model Training for Speculative Decoding

Feiye Huo, Jianchao Tan, Jiahao Liu, Zixu Jiang, Jiacheng Li, Jingang Wang, Xunliang Cai, Shengli Sun

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RepSpec, a structural re-parameterized training scheme for speculative decoding draft models (only the abstract was available).

Problem

Autoregressive LLM inference is memory-bound and slow; speculative decoding speedup is fundamentally limited by the quality of the small draft model.

Approach

Uses structural re-parameterization to train the draft model, so that extra training-time structure can be folded back into a plain draft architecture at inference. Details beyond the abstract are not available.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train draft with pure linear re-parameterization (Pre+Bypass)EAGLE-3 on LLaMA-3.1-8B, chain · avg of 6 tasksacceptance length: 3.35 -> 3.60Improved
train draft with pure linear re-parameterizationEAGLE-1 on LLaMA-3.1-8B, chain · avg of 6 tasksdecoding speed: 48 -> 52Improved
use hybrid (nonlinear bypass) instead of linearEAGLE-3 on LLaMA-3.1-8B, chain · avg of 6 tasksdecoding speed: 68 -> 65Hurt
use hybrid instead of linear on larger targetEAGLE-1 on LLaMA-2-13B, tree · avg of 6 tasksdecoding speed: 78 -> 80Improved
apply RepSpec Linear to MedusaMedusa on Vicuna-1.3-7B · avg of 6 tasksdecoding speed: 44 -> 46Improved
apply RepSpec Linear to Hydra acceptance lengthHydra on Vicuna-1.3-7B · avg of 6 tasksacceptance length: 3.59 -> 3.60No clear effect
re-parameterize the embedding layer (ablation)EAGLE-1 draft · 6 benchmarksacceptance length: MT 2.51 -> 2.55; GSM8K 2.74 -> 2.70Mixed
re-parameterize attention and MLP linear layers (ablation)EAGLE-1 draft · MT-bench validationacceptance rateImproved
use more than one Pre and one Bypass block (ablation)pure linear RepSpec · MT-bench validationacceptance rateNo clear effect
double decoder layers vs hybrid re-parameterization (ablation)EAGLE-1 draft · MT-benchacceptance length: 2.77 -> 2.79 (0.47B vs 0.40B params)Improved
choose ReLU instead of GeLU for bypass nonlinearity (ablation)hybrid RepSpec · MT-benchdecoding speed: tau 2.85 vs 2.85; speed 51 vs 50No clear effect
halve mid_feature in hybrid (ablation)hybrid RepSpec · MT-benchacceptance length: 2.85 -> 2.78; speed 51 -> 53Mixed

Experimental setup

Builds on: speculative decoding

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy