Research map / Image generation / Fast diffusion sampling
Self-Speculative Masked Diffusions
Andrew Campbell, Valentin De Bortoli, Jiaxin Shi, Arnaud Doucet
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| self-speculative sampling instead of standard masked diffusion | GPT2-scale MDM · OpenWebText 32 NFE | GPT2 NLL: 5.50 -> 5.28 | Improved |
| self-speculative sampling instead of standard masked diffusion | GPT2-scale MDM · OpenWebText 256 NFE | GPT2 NLL: 5.05 -> 5.02 | Improved |
| add output residual connection (ablation) | Speculative MDM · OpenWebText 32 NFE | GPT2 NLL: 5.36 -> 5.28 | Improved |
| add output residual connection (ablation) | Speculative MDM · OpenWebText 128 NFE | GPT2 NLL: 5.10 -> 5.05 | Improved |
| use 11 non-causal + 1 causal instead of 10 + 2 (ablation) | Speculative MDM · OpenWebText 32 NFE | GPT2 NLL: 5.34 -> 5.28 | Improved |
| use 11 non-causal + 1 causal instead of 10 + 2 (ablation) | Speculative MDM · OpenWebText 128 NFE | GPT2 NLL: 5.06 -> 5.05 | No clear effect |
| self-speculative sampling | MDM on text8 · text8 | NFE for spelling accuracy: greater than 2x reduction in NFE | Improved |
| self-speculative sampling with added causal block | frozen ESM2 150M on UniRef50 · UniRef50 | NFE for high pLDDT: ~2x speed-up for high pLDDT | Improved |
| self-speculative architecture overhead | MDM · standard settings | FLOPs: 0.98% increase in FLOPs | No clear effect |
| self-distillation (SDTT) vs self-speculative | MDM · OpenWebText 32 NFE | GPT2 NLL: 3.70 vs 5.28 | Mixed |
Closest papers by meaning
- Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.91)
- Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking (NeurIPS 2025 · similarity 0.90)
- Accelerated Diffusion Models via Speculative Sampling (ICML 2025 · similarity 0.90)
- Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling (ICLR 2025 · similarity 0.90)
- Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion (NeurIPS 2025 · similarity 0.90)
- Learn from Your Mistakes: Self-Correcting Masked Diffusion Models (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
