Research map / Image generation / Fast diffusion sampling
Beyond Masks: Efficient, Flexible Diffusion Language Models via Deletion-Insertion Processes
Fangyu Ding, Ding Ding, Sijin Chen, Kaibo Wang, Peng Xu, Zijin Feng, Haoli Bai, Kai Han, Youliang Yan, Binhang Yuan, Jiacheng Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace masking with deletion-insertion and train for FLOPs-aligned 800K steps (DID-F) vs RADD | RADD small, OWT · zero-shot WikiText | perplexity: 38.27 -> 36.91 | Improved |
| replace masking with deletion-insertion, steps-aligned (DID-S) vs RADD | RADD small, OWT · zero-shot PTB | perplexity: 108.79 -> 115.37 | Hurt |
| DID-F vs RADD on medium models | RADD medium, OWT · zero-shot PTB | perplexity: 81.05 -> 87.09 | Hurt |
| use DID instead of RADD at few denoising steps (16) | RADD small, OWT fixed length · generative PPL, 16 steps | generative PPL: 284.78 -> 158.93 | Improved |
| use DID instead of RADD at many denoising steps (512) | RADD small, OWT fixed length · generative PPL, 512 steps | generative PPL: 84.00 -> 86.04 | Hurt |
| use DID instead of RADD in variable-length setting | RADD small, Stories · generative PPL, 64 steps | generative PPL: 81.92 -> 22.78 | Improved |
| use DID instead of RADD (removing MASK tokens) for inference | RADD small, OWT · inference time, 512 steps | speedup: 1.58x | Improved |
| use DID instead of RADD in variable-length setting for training | RADD large, Stories · training time | speedup: 2.58x -> 3.42x with model size | Improved |
| train DID with DISE objective instead of DICE (fixed-length simplification) (ablation) | DID · zero-shot language modeling | perplexity | Hurt |
| train RADD at padding length 512 instead of 1024 (ablation) | RADD small, Stories · variable-length generation | speed and quality: ~1.59x slower than DID | Hurt |
Closest papers by meaning
- Theoretical Benefit and Limitation of Diffusion Language Model (NeurIPS 2025 · similarity 0.93)
- Scaling up Masked Diffusion Models on Text (ICLR 2025 · similarity 0.93)
- Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling (ICLR 2025 · similarity 0.92)
- Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.91)
- Looped Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
- Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
