Research map / Image generation / Fast diffusion sampling

Systematic Hazard Sampling: Minimal-Variance Inference for Discrete Diffusion and Flow Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Fast diffusion ODE solvers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Systematic Hazard Sampling (SHS), a training-free inference rule for uniform-noise discrete diffusion/flow models that replaces independent per-step Bernoulli edit decisions with unit-spaced thresholds on cumulative jump mass under a single random phase per position, reducing jump-count variance and improving generative perplexity.

Problem

Step-based samplers for uniform-noise discrete diffusion and flow models make independent Bernoulli edit decisions per step, producing Poisson-binomial jump-count variance that grows with NFE and causes under-editing and over-editing failures.

Approach

Couple per-step edit decisions through cumulative jump mass by triggering an edit whenever the cumulative mass crosses a unit-spaced threshold offset by one random phase per position; this preserves per-step marginals and expected jump count, caps count variance at 1/4, and leaves destination kernels unchanged.

Key results

• SHS lowers Gen. PPL at every tested (model, NFE) budget, by up to 26% on GIDD (NFE 256) and 17% on DFM (NFE 128).
• On UDLM, SHS at NFE=128 (175.7 Gen. PPL) beats the standard sampler at NFE=1024 (181.6).
• Empirical jump-count variance under SHS stays below 0.125-0.134 across NFE 4-128, while standard sampler variance grows from 0.344 to 3.087 on UDLM.

Surprising findings

• Adaptive top-1 scheduling, which also decides when to edit, yields 2-5x worse Gen. PPL than the standard sampler.
• Sharing one phase across positions raises UDLM Gen. PPL at NFE=128 from 174.2 to 213.5, showing per-position independence matters.

Limitations

• SHS is an inference-time integrator, not an exact simulator of the state-dependent CTMC/DTMC process.
• Quality cost of count deviation is mainly on the under-edit side; biasing the jump count is not explored.
• 3B model results use a single seed; experiments limited to text language models with no code benchmarks.

Experimental setup

Datasets: LM1B OpenWebText

Benchmarks: generative perplexity (GPT2-large) Distinct-n Self-BLEU MAUVE MNIST quantized generation

Metrics: Gen. PPL entropy Distinct-1/2/3 Self-BLEU MAUVE jump-count variance

Baselines: Standard step-based Bernoulli sampler Adaptive top-1 scheduling Nucleus sampling Temperature sampling

Builds on: D3PM SEDD UDLM GIDD discrete flow matching

Closest papers by meaning

• Leveraging Drift to Improve Sample Complexity of Variance Exploding Diffusion Models (NeurIPS 2024 · similarity 0.89)
• On the Error Correcting Effects of Stochasticity in Discrete Diffusion (NeurIPS 2026 · similarity 0.88)
• Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data (NeurIPS 2026 · similarity 0.88)
• O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.87)
• Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing (NeurIPS 2025 · similarity 0.87)
• Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.87)