Research map / Image generation / Fast diffusion sampling
Likelihood-Based Diffusion Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add learned noise schedule (ablation) | Plaid · OpenWebText2 val | NLL bound (lower is better): 4.17 -> 3.89 | Helped |
| add learned embeddings (ablation) | Plaid · OpenWebText2 val | NLL bound (lower is better): 4.54 -> 3.89 | Helped |
| add categorical reparameterization (ablation) | Plaid · OpenWebText2 val | NLL bound (lower is better): 4.25 -> 3.89 | Helped |
| add output prior (ablation) | Plaid · OpenWebText2 val | NLL bound (lower is better): 3.95 -> 3.89 | Helped |
| add learned conditional likelihood (ablation) | Plaid · OpenWebText2 val | NLL bound (lower is better): 4.03 -> 3.89 | Helped |
| add self-conditioning (ablation) | Plaid · OpenWebText2 val | NLL bound (lower is better): 3.98 -> 3.89 | Helped |
| doubling compute (0.5x -> full) | Plaid · OpenWebText2 val | NLL bound: 4.01 -> 3.89 | Helped |
| Plaid at half compute versus CDCD reimplementation | Plaid · OpenWebText2 val | NLL bound: 4.23 -> 4.01 | Helped |
| likelihood (VLB) weighting instead of heuristic schedule A (ablation) | Plaid · children's stories human eval | win rate of schedule A: 0.449 | Helped |
| likelihood (VLB) weighting instead of heuristic schedule B (ablation) | Plaid · children's stories human eval | win rate of schedule B: 0.457 | Helped |
| scale to Plaid 1B versus GPT-2 124M | Plaid · WikiText103 | perplexity: 35.92 -> 28.28 | Helped |
| truncate 3% of training examples to random lengths (ablation) | Plaid · sequence-length generalization | full-length performance | No clear effect |
Closest papers by meaning
- Scaling Diffusion Language Models via Adaptation from Autoregressive Models (ICLR 2025 · similarity 0.94)
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models (ICLR 2025 · similarity 0.92)
- Diffusion Beats Autoregressive in Data-Constrained Settings (NeurIPS 2025 · similarity 0.92)
- Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.92)
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.92)
- Continuous Diffusion Model for Language Modeling (NeurIPS 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.