Research map / Large language models / Language, safety and interpretability
Parallel Token Generation for Language Models
Felix Draxler, Justus Will, Farrin Marouf Sofian, Theofanis Karaletsos, Sameer Singh, Stephan Mandt
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition draft tokens on auxiliary variables instead of independent prediction (ablation) | O-PTP draft (TinyLlama-1.1B) · CodeContests speculative decoding | accepted tokens per step: 6.2 -> 7.0 | Improved |
| use parallel instead of sequential verification (ablation) | O-PTP · SpecBench | accepted tokens per step: 4.21 -> 2.89 | Hurt |
| distill O-PTP versus prior speculative methods | O-PTP (Vicuna-7B) · SpecBench task average | wall-clock speedup: 2.40 vs 2.34 (SAMD) | Improved |
| train C-PTP from data alone versus autoregressive model | C-PTP · Taxi pickup sequences | perplexity: 19.88 vs 19.81 | No clear effect |
| use structured (ar + lin) auxiliary embedding instead of fully learned (fl + NN) (ablation) | O-PTP · Taxi | average correct tokens: 35.9 -> 45.4 | Improved |
| use O-PTP instead of C-PTP (ablation) | PTP · Taxi | average correct tokens: 40.4 -> 45.4 | Improved |
| increase number of parallel proposal sequences M (ablation) | O-PTP · Taxi N=100 | average correct tokens: 45.36 -> 59.79 | Improved |
| limit tokens predicted per call N (ablation) | O-PTP · Taxi | average correct tokens: 45.36 (N=100) vs 13.90 (N=16) | Hurt |
| vary Beta parameter b for sampling training auxiliaries (ablation) | O-PTP · Taxi N=16 | average correct tokens: 12.9 / 13.9 / 13.8 | No clear effect |
| PTP draft instead of autoregressive draft at equal size | PTP draft · CodeContests speculative decoding | speedup | Improved |
| sample training sequences from teacher vs dataset (ablation) | O-PTP kl · Taxi N=16 | average correct tokens: 29 -> 40 | Improved |
Closest papers by meaning
- Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows (NeurIPS 2025 · similarity 0.91)
- Better & Faster Large Language Models via Multi-token Prediction (ICML 2024 · similarity 0.91)
- Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models (ICLR 2025 · similarity 0.91)
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time (ICLR 2025 · similarity 0.90)
- MARS: Enabling Autoregressive Models Multi-Token Generation (NeurIPS 2026 · similarity 0.89)
- From Bytes to Ideas: Language Modeling with Autoregressive U-Nets (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
