Research map / Architectures and efficiency / Attention and transformers
Blockwise Parallel Decoding for Deep Autoregressive Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use blockwise parallel decoding with k = 8 | Transformer on newstest2014 · newstest2014 | wall-clock speedup: 3.31x | Helped |
| Increase block size from k = 2 to k = 8 | Blockwise parallel decoding for Transformer · newstest2014 | BLEU: 28.95 -> 27.88 | Hurt |
| Use distilled rather than original training data (ablation) | Blockwise decoding, frozen Transformer, k = 8 · newstest2013 | BLEU: 25.82 -> 26.25 | Helped |
| Fine-tune the base parameters (ablation) | Blockwise decoding with original training data, k = 8 · newstest2013 | mean accepted block size: 1.76 -> 3.01 | Helped |
| Fine-tune the base parameters (ablation) | Blockwise decoding with original training data, k = 8 · newstest2013 | BLEU: 25.82 -> 24.27 | Hurt |
| Combine distillation and fine-tuning (ablation) | Blockwise decoding, k = 8 · newstest2013 | mean accepted block size: 1.91 -> 4.69 | Helped |
| Allow proposals from the scoring model's top-2 instead of requiring an exact greedy match (ablation) | Blockwise decoding, distillation and fine-tuning, k = 8 · newstest2013 | mean accepted block size: 4.69 -> 5.34 | Helped |
| Allow proposals from the scoring model's top-2 instead of requiring an exact greedy match (ablation) | Blockwise decoding, distillation and fine-tuning, k = 8 · newstest2013 | BLEU: 26.11 -> 25.71 | Hurt |
| Require a minimum block size of k = 2 or k = 3 (ablation) | Blockwise decoding for machine translation · newstest2013 | BLEU | Hurt |
| Allow distance-based approximate acceptance with epsilon = 2 (ablation) | Image Transformer with frozen base parameters, k = 10 · CelebA development set | mean accepted block size: 1.10 -> 1.40 | Helped |
| Fine-tune base parameters (ablation) | Image Transformer with exact acceptance, k = 10 · CelebA development set | mean accepted block size: 1.10 -> 2.04 | Helped |
| Combine fine-tuning and distance-based approximate acceptance (ablation) | Image Transformer, k = 10 · CelebA development set | mean accepted block size: 1.10 -> 6.79 | Helped |
Closest papers by meaning
- Accelerating Blockwise Parallel Language Models with Draft Refinement (NeurIPS 2024 · similarity 0.91)
- Break the Sequential Dependency of LLM Inference Using Lookahead Decoding (ICML 2024 · similarity 0.87)
- DFlash: Block Diffusion for Flash Speculative Decoding (ICML 2026 · similarity 0.87)
- Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.87)
- When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models (ICML 2024 · similarity 0.87)
- AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.