Research map / Large language models / Language, safety and interpretability
Parallel and early-exit decoding
Speed up decoding through parallel, layer-adaptive, and diffusion-based generation.
100 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2021–2026) · an approach in Language, safety and interpretability, keyword cluster “prompt · uncertainty · shot”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
18 papers in 2023–24 and 80 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 2.6% to 4.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2021 · Adaptive Beam Search Decoding for Discrete Keyphrase Generation (AAAI 2021 · 7 citations)
- 2022 · Confident Adaptive Language Modeling (NeurIPS 2022 · 39 citations)
- 2023 · Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models (ICML 2023 · 1 citation)
- 2024 · Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models (AAAI 2024 · 34 citations)
- 2025 · Adaptive Draft-Verification for Efficient Large Language Model Decoding (AAAI 2025 · 2 citations)
- 2026 · DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention (ICML 2026 · 0 citations)
Earliest papers here
- Adaptive Beam Search Decoding for Discrete Keyphrase Generation (AAAI 2021 · 7 citations)
- Confident Adaptive Language Modeling (NeurIPS 2022 · 39 citations)
- Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models (ICML 2023 · 1 citation)
What has been tried
39 papers here report 434 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 6%, hurt 25%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Vary a from 0.2 to 0.9 with b=0.3 | HumanEval · pass@15 | 32.9% to 37.8% | Helped | Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models (AAAI 2024 · 34 citations) |
| Add tree attention to Medusa heads | MT-Bench writing and roleplay · speedup | \sim1.5x -> \sim1.9x | Helped | Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads (ICML 2024 · 13 citations) |
| ignore loss on <mention n> text spans | CoNLL03 · micro F-score | 92.52 -> 92.01 | Hurt | PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition (NeurIPS 2024 · 5 citations) |
| replace on-policy sampling with off-policy sampling | Dolly Eval · score | 26.37 -> 26.12 | Hurt | DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024 · 4 citations) |
| Use lookahead k=0 instead of k=∞ | GSM8K · accuracy | 0.418 -> 0.308; 0.157 -> 0.0 | Hurt | Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation (ICML 2024 · 3 citations) |
| set filtered-set reduction R to 8 rather than R=64 | AdvBench validation · ASR | 60.0 -> 85.0 | Helped | Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling (NeurIPS 2024 · 3 citations) |
| use the adaptive strategy instead of the non-adaptive strategy | MT-Bench (Vicuna-7B) · average accept length | Helped | Adaptive Draft-Verification for Efficient Large Language Model Decoding (AAAI 2025 · 2 citations) | |
| add the “think in parallel” prompt | AIME24 · pass@1 | 52.1 -> 53.8 | Helped | Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation (NeurIPS 2025 · 1 citation) |
| Use left-to-right decoding instead of entropy-based decoding | GSM8K · accuracy | 0.804 -> 0.832 | Helped | Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · 1 citation) |
| Add keyword-and-topic instruction to the BOLT prompt | Topic-guided generation · Success Rate | 99.1% -> 99.7% | Helped | Controlled LLM Decoding via Discrete Auto-regressive Biasing (ICLR 2025 · 1 citation) |
| Use n-grams 2–4 instead of n-grams 2–6 | OpenWebText test split, best of 3 drafts · average perplexity | 5.17 -> 5.18 | No clear effect | Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass (NeurIPS 2024 · 1 citation) |
| Add Relative Retrieval Confidence to two-stage kNN-LM | WikiText-103 validation · ROUGE-1 | 20.1 -> 24.7 | Helped | Nearest Neighbor Speculative Decoding for LLM Generation and Attribution (NeurIPS 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 33 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- HumanEval · 10 of 33 papers
- GSM8K · 9 of 33 papers
- MBPP · 6 of 33 papers
- MATH · 4 of 33 papers
- MT-Bench · 3 of 33 papers
- ShareGPT · 2 of 33 papers
- XSum · 2 of 33 papers
- AlpacaEval · 2 of 33 papers
Baselines
- autoregressive decoding · 3 of 33 papers
- Medusa · 3 of 33 papers
- Outlines · 2 of 33 papers
- XGrammar · 2 of 33 papers
- SynCode · 2 of 33 papers
- LookAhead · 2 of 33 papers
- EAGLE · 2 of 33 papers
- SEDD · 2 of 33 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 98 papers; from OpenAlex and the papers' affiliations).
- Google · 7 papers
- Meta · 5 papers
- Google DeepMind · 4 papers
- Peking University · 3 papers
- Microsoft · 3 papers
- Tsinghua University · 3 papers
- Carnegie Mellon University · 3 papers
- University of California, San Diego · 3 papers
- Massachusetts Institute of Technology · 3 papers
- Northwestern University · 2 papers
Use it
Download BibTeX of 23 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- In-context learning analysis (95 papers)
Elsewhere in the topic
- Uncertainty quantification for LLMs (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Safety alignment of LLMs (123 papers)
- Linguistic encoding in LLM representations (123 papers)
- Reasoning mechanisms and failure modes (114 papers)
- Speech and audio language models (111 papers)
- Sign and cross-lingual translation (96 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).