Research map / Large language models / Reasoning and chain of thought
PEARL: Parallel Speculative Decoding with Adaptive Draft Length
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PEARL, a parallel speculative decoding framework that runs draft and target models concurrently using pre-verify (verify first draft token early) and post-verify (keep drafting during verification) to get adaptive draft length, reaching up to 4.43x speedup over autoregressive decoding.
Problem
Speculative decoding suffers from a mutual waiting problem: the target model idles while the draft model guesses tokens and vice versa, worsened by a fixed draft length.
Approach
Pre-verify checks the first draft token using the target model on the prefix during drafting, and post-verify continues drafting during verification assuming full acceptance; switching between these two modes makes drafting and verification run in parallel with adaptive draft length, without any training.
Key results
- PEARL reaches up to 4.43x speedup over autoregressive decoding on HumanEval (CodeLlama 7B&70B) and up to 1.50x over vanilla speculative decoding.
- On MGSM with Llama 2 7B&70B, PEARL averages 3.95x versus 2.70x for speculative decoding.
- On MT-bench with Llama 2 7B&70B, PEARL averages 2.64x versus 1.89x for speculative decoding; with Llama 3.1 8B&70B it averages 3.59x versus 2.42x.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PEARL instead of the strongest listed baseline | CodeLlama and Llama/DeepSeek model pairs · HumanEval | walltime speedup: up to 4.43$\times$ vs 1.00$\times$ Auto Regressive; up to 1.50$\times$ vs Speculative Decoding | Helped |
| Use PEARL instead of Speculative Decoding | Llama 2 7B&70B · GSM8K and MGSM | walltime speedup: 2.48$\times$ -> 3.82$\times$ (GSM8K); 2.70$\times$ -> 3.95$\times$ (Avg.) | Helped |
| Use PEARL instead of Speculative Decoding | Llama 2 7B&70B · MT-Bench | walltime speedup: 1.89$\times$ -> 2.64$\times$ (Avg.) | Helped |
| Add pre-verify (ablation) | PEARL · HumanEval, CodeLlama 7B&34B | walltime speedup: 2.21$\times$ -> 2.35$\times$ ( +0.14 shown as ↓ 0.14 when removed) | Helped |
| Add pre-verify (ablation) | PEARL · HumanEval, CodeLlama 7B&70B | walltime speedup: 3.53$\times$ -> 3.79$\times$ (↓ 0.26 when removed) | Helped |
| Add pre-verify (ablation) | PEARL · HumanEval, DeepSeek 1.3B&33B | walltime speedup: 3.19$\times$ -> 3.48$\times$ (↓ 0.29 when removed) | Helped |
| Add pre-verify (ablation) | PEARL · GSM8K, Llama 2 7B&70B | walltime speedup: 2.51$\times$ -> 2.87$\times$ (↓ 0.26 when removed) | Helped |
| Add post-verify (ablation) | PEARL · HumanEval, CodeLlama 7B&34B | walltime speedup: 1.64$\times$ -> 2.35$\times$ (↓ 0.71 when removed) | Helped |
| Add post-verify (ablation) | PEARL · HumanEval, CodeLlama 7B&70B | walltime speedup: 2.57$\times$ -> 3.79$\times$ (↓ 1.22 when removed) | Helped |
| Add post-verify (ablation) | PEARL · HumanEval, DeepSeek 1.3B&33B | walltime speedup: 2.37$\times$ -> 3.48$\times$ (↓ 1.11 when removed) | Helped |
| Add post-verify (ablation) | PEARL · GSM8K, Llama 2 7B&70B | walltime speedup: 2.15$\times$ -> 2.87$\times$ (↓ 0.72 when removed) | Helped |
Surprising findings
- Post-verify contributes more to speedup than pre-verify because the evaluated model pairs are well aligned, so full-acceptance cases are common.
- Under a co-located (resource competition) setting, PEARL retains about 89-99% of its performance only when the draft and target models are split across GPUs with a pipeline-parallel scheme.
Limitations
- Parallel execution increases total forward passes of both models, raising power consumption.
- Resource competition when draft and target share a GPU degrades speedup; requires careful GPU scheduling.
- Only pre-verifies a single draft token; authors note that pre-verifying multiple tokens is future work.
Experimental setup
Datasets: HumanEval GSM8K MGSM MT-bench
Benchmarks: HumanEval GSM8K MGSM MT-bench
Metrics: walltime speedup ratio tokens per second mean accepted tokens
Baselines: Speculative Decoding Ouroboros Lookahead Decoding Assisted Generation
Builds on: speculative decoding speculative sampling lookahead decoding Ouroboros
Closest papers by meaning
- SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism (ICLR 2026 · similarity 0.93)
- TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees (NeurIPS 2026 · similarity 0.92)
- SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration (ICLR 2025 · similarity 0.92)
- SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.92)
- A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.92)
- Speculative Speculative Decoding (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.