Research map / Architectures and efficiency / Attention and transformers

P-EAGLE: Parallel-Drafting EAGLE with Scalable Training

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: Sparse attention for long-context LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P-EAGLE, which converts EAGLE-3 into a single-pass parallel multi-token drafter with a learnable shared hidden state, and makes long-context training feasible via pre-computed attention masks and within-sequence sequence partitioning, reporting 1.10x-1.36x vLLM speedups over AR EAGLE-3.

Problem

Parallel speculative drafting promises latency gains over autoregressive drafting, but its training cost grows quadratically with sequence length times the number of parallel prediction depths, making long-context (reasoning-length) training impractical.

Approach

Replace per-position hidden states with a learnable shared hidden state and mask token embedding so all K draft tokens come from one forward pass; pre-compute a maximum-length causal mask and slice it per example; partition a single sequence into segments with dependency-preserving assignment so gradients can be accumulated within one sequence.

Key results

• P-EAGLE (4-layer) matches or exceeds AR EAGLE-3 acceptance length on all 9 model-dataset pairs, averaging +4.5% on GPT-OSS 120B, +2.5% on GPT-OSS 20B, +2.0% on Qwen3-Coder 30B.
• vLLM OTPS speedups reach up to 1.36x on GPT-OSS 20B at C=2 and K=7, with 1.27x-1.36x for 20B at C=2.
• On GPT-OSS 120B, P-EAGLE trains at 20K context where ParallelSpec and PARD hit OOM at 8K+, with acceptance length 3.0 at 20K.

Surprising findings

• Unfrozen mask-token embeddings and a learnable shared hidden state outperform adding depth-specific or NTP-hidden context injection by 7-15%.
• Training with K_train=8 while inferring at K_infer=5 improves acceptance by about 4% over matched settings.

Limitations

• Speedups are modest for large models (1.04x-1.10x for GPT-OSS 120B at C=2) and shrink at higher concurrency (C=4).
• Slight slowdowns on Qwen 30B at K=3 (0.92x-0.98x) due to the deeper 4-layer drafter.
• Evaluation uses out-of-distribution benchmarks with acceptance length at K=5; comparisons to ParallelSpec and PARD are limited because those baselines OOM at 8K+ context.

Experimental setup

Datasets: UltraChat GSM-8K OpenCodeInstruct

Benchmarks: HumanEval MT-Bench GSM-8K

Metrics: acceptance length output tokens per second (OTPS) speedup

Baselines: AR EAGLE-3 ParallelSpec PARD

Builds on: EAGLE EAGLE-3 PARD ParallelSpec Training-Time Test

Closest papers by meaning

• D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting (NeurIPS 2026 · similarity 0.89)
• Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning (ICLR 2026 · similarity 0.88)
• Steering Pretrained Drafters During Speculative Decoding (AAAI 2026 · similarity 0.87)
• PARD: Accelerating LLM Inference with Low‑Cost PARallel Draft Model Adaptation (ICLR 2026 · similarity 0.87)
• EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test (NeurIPS 2025 · similarity 0.86)
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting (NeurIPS 2026 · similarity 0.86)