Research map / Image generation / Fast diffusion sampling
Fast Sampling via Discrete Non-Markov Diffusion Models with Predetermined Transition Time
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DNDM, a training-free non-Markov discrete diffusion sampler whose predetermined transition-time set cuts function evaluations, plus a continuous-time (infinite-step) variant DNDM-C.
Problem
Discrete diffusion models (D3PM-style multinomial and absorbing) need one neural network evaluation per step, making sampling slow; acceleration of discrete diffusion sampling is under-explored.
Approach
Replace the per-step noise with a time-invariant noise draw so each token has a single transition time; sampling the transition times upfront de-randomizes the reverse process and lets the model be evaluated only at the set of transition times, with the same marginals and a model trained under the Markov objective.
Key results
- On IWSLT14 absorbing diffusion at 1000 steps, DNDM-Absorb reaches 32.93 BLEU in 161.1 s versus RDM-Absorb's 31.91 BLEU in 4197.4 s.
- DNDM-C (infinite steps) on IWSLT14 multinomial reaches 31.89 BLEU (DNDM-Multi) and 33.44 BLEU with top-k (DNDM-k-Multi) in roughly 225-228 s.
- On text8 unconditional generation, DNDM cuts sampling time from 135.9 s to 31.1 s and perplexity from 1465.75 to 600.02.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DNDM-k-Multi instead of RDM-k-Multi | Multinomial diffusion sampling · IWSLT14 DE-EN (1000 steps) | BLEU: 32.64 -> 33.15 | Helped |
| Use DNDM-k-Multi instead of RDM-k-Multi | Multinomial diffusion sampling · WMT14 EN-DE (1000 steps) | BLEU: 25.82 -> 26.88 | Helped |
| Use DNDM-k-Multi instead of RDM-k-Multi | Multinomial diffusion sampling · WMT16 EN-RO (1000 steps) | BLEU: 33.67 -> 33.79 | Helped |
| Use DNDM-k-Absorb instead of RDM-k-Absorb | Absorbing diffusion sampling · IWSLT14 DE-EN (1000 steps) | BLEU: 34.60 -> 34.56 | Hurt |
| Use DNDM-k-Absorb instead of RDM-k-Absorb | Absorbing diffusion sampling · WMT14 EN-DE (1000 steps) | BLEU: 27.75 -> 27.82 | Helped |
| Use DNDM-k-Absorb instead of RDM-k-Absorb | Absorbing diffusion sampling · WMT16 EN-RO (1000 steps) | BLEU: 34.44 -> 34.38 | Hurt |
| Use DNDM-k-Multi instead of RDM-k-Multi | Multinomial diffusion sampling · IWSLT14 DE-EN, WMT14 EN-DE, and WMT16 EN-RO (1000 steps) | Time (s): 6321.3 -> 191.5; 8991.7 -> 265.5; 5601.0 -> 171.2 | Helped |
| Use DNDM-k-Absorb instead of RDM-k-Absorb | Absorbing diffusion sampling · IWSLT14 DE-EN, WMT14 EN-DE, and WMT16 EN-RO (1000 steps) | Time (s): 4205.9 -> 162.3; 4167.4 -> 179.1; 2718.7 -> 122.7 | Helped |
| Use DNDM instead of vanilla sampling | Multinomial diffusion language modeling · text8 | Perplexity: 1,465.75 -> 600.02 | Helped |
| Use DNDM instead of vanilla sampling | Multinomial diffusion language modeling · enwik8 | Perplexity: 801.78 -> 556.78 | Helped |
| Use DNDM instead of vanilla sampling | Multinomial diffusion language modeling · text8 | Time (s): 135.9 -> 31.1 | Helped |
| Use DNDM instead of vanilla sampling | Multinomial diffusion language modeling · enwik8 | Time (s): 602.8 -> 47.4 | Helped |
Surprising findings
- Infinite-step DNDM-C does not consistently beat the 1000-step DNDM variant; on WMT14 BLEU drops for some settings (e.g., 24.79 vs 25.66 for DNDM-Multi).
Limitations
- Evaluated only on text with non-autoregressive models; audio, image and autoregressive GPT-style settings not explored.
- Infinite-step sample quality is not guaranteed to beat the 1000-step accelerated version, possibly due to accumulated estimation error.
- Hyperparameter regions with poor sample quality are not explored.
Experimental setup
Datasets: IWSLT14 DE-EN WMT14 EN-DE WMT16 EN-RO text8 enwik8
Benchmarks: IWSLT14 DE-EN WMT14 EN-DE WMT16 EN-RO text8 enwik8
Metrics: BLEU wall-clock time (s) perplexity NFE
Baselines: RDM RDM-k D3PM vanilla sampling (multinomial) D3PM absorbing
Builds on: D3PM DDIM RDM (reparameterized diffusion model) ARDM
Closest papers by meaning
- The Diffusion Duality (ICML 2025 · similarity 0.91)
- Discrete Diffusion Models: Novel Analysis and New Sampler Guarantees (NeurIPS 2025 · similarity 0.91)
- Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling (ICLR 2025 · similarity 0.91)
- Constrained Discrete Diffusion (NeurIPS 2025 · similarity 0.91)
- Breaking AR’s Sampling Bottleneck: Provable Acceleration via Diffusion Language Models (NeurIPS 2025 · similarity 0.90)
- Accelerated Diffusion Models via Speculative Sampling (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.