Research map / Large language models / Reasoning and chain of thought
AdaSpec: Adaptive Multilingual Speculative Decoding with Self-Synthesized Language-Aware Training and Vocabulary Simplification
Dinh-Truong Do, Nguyen-Khang Le, Le-Minh Nguyen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaSpec, a multilingual speculative decoding method that pairs a lightweight drafter with self-synthesized language-aware training and vocabulary simplification to address static-vocabulary and language-data gaps.
Problem
Speculative decoding for LLMs works well in English but struggles in multilingual settings due to static vocabularies and a lack of language-specific instruction data.
Approach
Adapts speculative decoding to multiple languages by self-synthesizing language-aware training data and simplifying the drafter's vocabulary, rather than relying on static vocabularies and external language-specific instruction data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the drafter on target-language synthesized instruction data instead of English data (ablation) | AdaSpec (Llama3-8B) · Multi-SpecBench German | speedup ratio: 1.30 -> 1.83 | Improved |
| train the drafter on target-language data (ablation) | AdaSpec (Llama3-8B) · Multi-SpecBench Japanese | speedup ratio: 1.25 -> 1.91 | Improved |
| use language-specific simplified vocabulary instead of the full vocabulary (ablation) | AdaSpec (Llama3-8B) · Multi-SpecBench German | speedup ratio: 1.58 -> 1.83 | Improved |
| use adaptive vocabulary size instead of a fixed size (ablation) | AdaSpec (Llama3-8B) · Multi-SpecBench German | speedup ratio: 1.71 -> 1.83 | Improved |
| use language-specific vocabulary instead of full vocabulary (ablation) | AdaSpec (Llama3-8B) · Multi-SpecBench German | compression ratio tau: 2.79 -> 2.71 | No clear effect |
| replace EAGLE-2 with AdaSpec for multilingual decoding | Llama3-8B-Instruct · Multi-SpecBench German | speedup ratio: 1.15 -> 1.83 | Improved |
| increase adaptive window size from 5 to 15 (u=1, z=0) (ablation) | AdaSpec · Multi-SpecBench multilingual mix | throughput (tokens/s): 135.22 -> 136.70 | Improved |
| add larger step size (u=3) and offset (z=1) (ablation) | AdaSpec (w=10) · Multi-SpecBench multilingual mix | throughput (tokens/s): 136.30 -> 138.30 | Improved |
| increase window to 15 at u=5, z=2 (ablation) | AdaSpec · Multi-SpecBench multilingual mix | throughput (tokens/s): 139.93 -> 138.91 | Hurt |
| use EAGLE-2 and FR-Spec without language adaptation on small model | Llama3.2-1B-Instruct · Multi-SpecBench multilingual | speedup vs autoregressive | Hurt |
| implement in C/CUDA instead of Python | speculative decoding methods · Multi-SpecBench monolingual | speedup | Improved |
Experimental setup
Builds on: speculative decoding
Closest papers by meaning
- Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies (ICML 2025 · similarity 0.91)
- SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.91)
- AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders (NeurIPS 2025 · similarity 0.90)
- Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling (ICLR 2026 · similarity 0.89)
- CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs (NeurIPS 2025 · similarity 0.89)
- A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
