Research map / Image generation / Fast diffusion sampling

Classification Diffusion Models: Revitalizing Density Ratio Estimation

NeurIPS 2024 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes classification diffusion models (CDMs), a density-ratio-estimation generative model whose noise-level classifier is trained with CE plus an MSE loss on its gradient, yielding image generation beyond MNIST and single-pass exact likelihood.

Problem

Density ratio estimation (DRE) methods can output likelihoods in one pass but fail to model complex high-dimensional images such as CIFAR-10 and CelebA, while diffusion models require many function evaluations for likelihood.

Approach

An analytical connection shows the MMSE denoiser for white Gaussian noise equals the gradient of the cross-entropy-optimal noise-level classifier's log-ratio; training the classifier with CE plus an MSE loss on this gradient avoids the density-chasm and global-miscalibration problems, enabling image generation and single-step exact likelihood.

Key results

• CDM with OT scheduler achieves single-step NLL of 2.89 bits/dim on CIFAR-10 with FID 7.07 at 1000 sampling steps, state of the art among single-NFE methods.
• Training with both CE and MSE losses gives FID 7.56 and NLL 3.38 on CIFAR-10, versus FID 329 for CE-only training.
• CDM denoising MSE surpasses pre-trained DDMs at high noise levels and is comparable at lower noise levels.

Measured effects reported in the paper

Surprising findings

• A classifier trained with CE alone achieves reasonable classification accuracy yet produces very poor generation (FID 329), showing CE alone does not learn globally correct probability ratios.

Limitations

• CDM steps require a forward and a backward pass, so sampling is more computationally expensive than DDMs.
• Architecture is the DDPM U-Net, not optimized for CDMs; the authors leave architecture exploration to future work.
• NLL remains comparable rather than superior to multi-step DDM likelihood methods at the default scheduler.

Experimental setup

Datasets: CIFAR-10 CelebA

Benchmarks: CIFAR-10 CelebA 64x64

Metrics: FID NLL (bits/dim) MSE classification accuracy CE

Baselines: DDPM DDIM DPM-Solver TRE NCE

Builds on: telescoping density-ratio estimation denoising diffusion probabilistic models noise contrastive estimation Tweedie's formula

Closest papers by meaning

• UDPM: Upsampling Diffusion Probabilistic Models (NeurIPS 2024 · similarity 0.88)
• Training Diffusion-based Generative Models with Limited Data (ICML 2025 · similarity 0.88)
• Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.88)
• Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training (NeurIPS 2024 · similarity 0.87)
• BGDB: Bernoulli-Gaussian Decision Block with Improved Denoising Diffusion Probabilistic Models (AAAI 2025 · similarity 0.87)
• Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.87)