Research map / Image generation / Fast diffusion sampling

Bridging discrete and continuous state spaces: Exploring the Ehrenfest process in time-continuous diffusion models

ICML 2024 · Image generation › Fast diffusion sampling · approach: Diffusion theory and learned samplers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Links time-reversed Markov jump processes on discrete states (Ehrenfest process) to score-based SDE diffusion models, proposing a conditional-expectation loss and showing convergence to the reversed Ornstein-Uhlenbeck process.

Problem

Generative modeling with time-continuous Markov jump processes on discrete state spaces lacks an exact correspondence to state-continuous score-based diffusion models, and learning time-reversed jump rates is hard.

Approach

Uses the Ehrenfest process (a birth-death Markov jump process) as forward process; shows its scaled version converges to an Ornstein-Uhlenbeck process and its time-reversal to the reversed OU SDE, so the reverse rates' conditional expectations map to score functions; proposes an L2 conditional-expectation loss analogous to denoising score matching.

Key results

• On CIFAR-10 trained from scratch, the Ehrenfest model with L_OU reaches IS 9.50 and FID 5.08 (Taylor loss IS 9.66, FID 5.12).
• Fine-tuning a pretrained continuous U-Net with the L_OU discrete loss gives FID 6.63 (IS 9.14), compared with D3PM Gauss FID 7.34 and D3PM Absorbing FID 30.97.
• The L_Gauss variant did not produce satisfying results on CIFAR-10 and had convergence issues.

Measured effects reported in the paper

Surprising findings

• Transfer learning from a pretrained continuous DDPM without further training yields good discrete samples (FID 11.57 on CIFAR-10).
• The Gaussian-approximation loss L_Gauss, which performed best on the 2D toy example, failed to converge on CIFAR-10.

Limitations

• Convergence of the time-reversed jump process to the reversed SDE is established only heuristically (conjectured), with formal results left to future work.
• Results on CIFAR-10 were obtained without extensive hyperparameter or sampler tuning.
• Requires a separate approximation for each state-dependent conditional expectation per dimension in the general case.

Experimental setup

Datasets: MNIST CIFAR-10

Benchmarks: CIFAR-10

Metrics: Inception Score FID

Baselines: D3PM tau-LDR DDPM

Builds on: denoising diffusion probabilistic models discrete denoising diffusion (D3PM) score-based SDE

Closest papers by meaning

• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis (ICLR 2025 · similarity 0.88)
• Piecewise deterministic generative models (NeurIPS 2024 · similarity 0.88)
• Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.88)
• ABC: Any-Subset Autoregression via Non-Markovian Diffusion Bridges in Continuous Time and Space (NeurIPS 2026 · similarity 0.87)
• Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models (NeurIPS 2025 · similarity 0.87)
• O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.87)