Research map / Image generation / Fast diffusion sampling
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping
Zijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| early-skipping (ES-dLLM) vs DualCache | ES-dLLM · LLaDA-8B GSM8K | throughput (TPS): 112.15 -> 143.93 | Improved |
| ES-dLLM vs vanilla on accuracy | ES-dLLM · LLaDA-8B MATH | accuracy: 28.14 -> 27.24 | Hurt |
| more frequent prompt cache refresh (ES-dLLM*) (ablation) | ES-dLLM · LLaDA-8B BBH | accuracy: 54.51 -> 56.66 | Improved |
| more frequent prompt cache refresh (ES-dLLM*) (ablation) | ES-dLLM · LLaDA-8B BBH | throughput (TPS): 159.89 -> 133.48 | Hurt |
| use only confidence (alpha=1) instead of mixing with variation (ablation) | ES-dLLM · LLaDA-8B multiple benchmarks | accuracy | Hurt |
| use variation only (alpha=0) (ablation) | ES-dLLM · LLaDA-8B MATH | accuracy | Improved |
| swap hidden states for key/value as variation indicator (ablation) | ES-dLLM · LLaDA-8B | accuracy | No clear effect |
| skip earlier layer (layer 0 vs layer 16) (ablation) | ES-dLLM · LLaDA-8B MATH | speedup vs DualCache: 1.60x vs 1.22x speedup; 26.76 vs 27.52 score | Mixed |
| larger skip ratio at layer 8 (0.25 to 0.75) (ablation) | ES-dLLM · LLaDA-8B MATH | speedup vs DualCache: 1.13x -> 1.70x; score 27.58 -> 27.26 | Mixed |
| two skip positions instead of one at layer 4 (equal FLOPs) (ablation) | ES-dLLM · LLaDA-8B HumanEval | pass@1: 35.98 -> 37.8 | Improved |
| three skip positions instead of two (equal FLOPs) (ablation) | ES-dLLM · LLaDA-8B HumanEval | pass@1: 37.8 -> 36.59 | Hurt |
| combine with parallel decoding | ES-dLLM · Dream-7B MBPP | throughput (TPS): 829.71 -> 1469.15 | Improved |
Closest papers by meaning
- dKV-Cache: The Cache for Diffusion Language Models (NeurIPS 2025 · similarity 0.91)
- SemanticDLM+: Improving Diffusion LLMs through Bias-variance Trade-off in Transition Kernel Design (NeurIPS 2026 · similarity 0.90)
- DAWN: Dependency-Aware Fast Inference for Diffusion LLMs (NeurIPS 2026 · similarity 0.90)
- Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models (ICML 2026 · similarity 0.90)
- SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.90)
- EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
