Research map / Image generation / Fast diffusion sampling

Conservation Laws for Diffusion Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Diffusion theory and learned samplers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives a mismatched GEXIT conservation law showing data-model cross-entropy equals an integral of local marginal-posterior derivatives along the noise path for masked, uniform, and Gaussian diffusion, and validates it on Markov sources, text8, and CIFAR-10.

Problem

Diffusion models are trained with denoising objectives rather than exact likelihood; it is unclear how to characterize the induced data likelihood (cross-entropy) uniformly across discrete and continuous memoryless noise processes.

Approach

Extends GEXIT/EXIT identities to mismatched posteriors, giving an exact area-theorem conservation law for cross-entropy along any admissible noise path; the integrand depends only on learned marginal posteriors, and training reduces to a channel-shared token-level NLL with channel-tilted logits.

Key results

• On text8, masked diffusion achieves 1.424 BPC versus 1.513 for uniform and 2.087 for Gaussian (no external data).
• On CIFAR-10, Gaussian diffusion gives 3.24 bpd likelihood (masked 3.31, uniform 3.66) and FID 26.66 with IS 7.34.
• On synthetic Markov sources, masked and uniform GEXIT estimates stay close to the true entropy rate while Gaussian estimates lie above it with a gap growing with alphabet size.

Surprising findings

• Although the conservation law is channel-independent in the exact case, finite-capacity denoisers yield substantially different likelihood estimates across noise types.
• Gaussian noise gives the best CIFAR-10 likelihood but the worst text8 BPC (2.087).

Limitations

• Theory assumes memoryless degraded channels and posteriors factoring through the known channel; the marginal-only locality holds infinitesimally, not for finite-step sampling.
• Experiments limited to synthetic Markov, byte-level text8, and CIFAR-10; single training seed per objective.
• Gaussian formulation computes means over discrete support, which may be expensive for large vocabularies.

Experimental setup

Datasets: synthetic first-order Markov sources text8 CIFAR-10

Benchmarks: text8 CIFAR-10

Metrics: bits per character (BPC) bits per dimension (bpd) FID Inception Score entropy rate estimate

Baselines: D3PM SEDD MDLM MD4 EDLM DDPM Improved DDPM Score SDE

Builds on: GEXIT I-MMSE D3PM Verdú mismatched estimation

Closest papers by meaning

• Information-Theoretic Discrete Diffusion (NeurIPS 2025 · similarity 0.92)
• O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.90)
• Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.90)
• Information Theoretic Learning for Diffusion Models with Warm Start (NeurIPS 2025 · similarity 0.89)
• Generalization through variance: how noise shapes inductive biases in diffusion models (ICLR 2025 · similarity 0.89)
• Feature Information Dynamics in Diffusion (NeurIPS 2026 · similarity 0.88)