Research map / Image generation / Fast diffusion sampling
DPad: Efficient Diffusion Language Models with Suffix Dropout
Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai Helen Li, Yiran Chen
This paper tried a common change: Add dropout
add DPad suffix dropout · GSM8K (4-shot) · strict-match accuracy: 37.38 -> 63.84 Improved
What 795 accepted papers report for this change
- 549 improved
- 87 no clear effect
- 12 mixed
- 147 hurt
- Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference ICML 2026 · apply layer dropout to whole transformer blocks instead of attention and FFN sub-blocks separately · 20 TPP val loss, 271M · val loss: 2.377 -> 2.367Improved
- Quantifying the Uncertainty of Foundation Models with Singular Value Ensembles ICML 2026 · use MC Dropout · vision and SST-2 · accuracy: 86.3 -> 76.6 on Flowers102Hurt
- Motion Planning in Compressed Representation Spaces ICML 2026 · add nested dropout during training · WOMD guided generation · goal FDE: 4.145 -> 1.104Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DPad suffix dropout | LLaDA-Instruct · GSM8K (4-shot) | strict-match accuracy: 37.38 -> 63.84 | Improved |
| add DPad suffix dropout | LLaDA-Instruct · MATH (4-shot) | strict-match accuracy: 8.42 -> 28.04 | Improved |
| add DPad suffix dropout | LLaDA-Instruct · MBPP (3-shot) | pass@1: 15.00 -> 40.40 | Improved |
| add DPad suffix dropout | LLaDA-1.5 · GSM8K (4-shot) | flexible-match accuracy: 80.59 -> 80.14 | No clear effect |
| add DPad suffix dropout | Dream-Base · GSM8K (4-shot) | strict-match accuracy: 74.37 -> 75.06 | No clear effect |
| add DPad on top of parallel decoding | LLaDA-Instruct · MBPP (3-shot) | latency speedup: 4.36× -> 10.32× | Improved |
| add DPad on top of parallel decoding | Dream-Base · HumanEval (0-shot) | latency speedup: 2.01× -> 7.01× | Improved |
| add DPad to vanilla Dream at 2048 tokens | Dream-Base · HumanEval (2048) | accuracy: 7.32% accuracy degradation | Hurt |
| extend generation length to 1024 tokens (1-shot) (ablation) | LLaDA-1.5 · GSM8K | latency speedup of DPad: 1.51× -> 20.3× | Improved |
| use Gaussian instead of uniform dropout (ablation) | LLaDA-1.5 · GSM8K (window/budget sweep) | flexible and strict-match accuracy | Improved |
| add DPad at 1-shot | LLaDA-1.5 · GSM8K (1-shot) | strict-match accuracy: baseline ~60% -> ~50% vs DPad ~78% -> ~74% | Improved |
| use k=1 Gaussian decay at 25.0% density (ablation) | LLaDA-Instruct · GSM8K subset (50 samples) | flexible-match accuracy | Hurt |
Closest papers by meaning
- Accelerating Diffusion Language Models via Structured Suffix Modeling (NeurIPS 2026 · similarity 0.93)
- DAWN: Dependency-Aware Fast Inference for Diffusion LLMs (NeurIPS 2026 · similarity 0.91)
- dKV-Cache: The Cache for Diffusion Language Models (NeurIPS 2025 · similarity 0.90)
- Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.90)
- Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.90)
- TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
