Research map / Image generation / Fast diffusion sampling
Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Contrastive Diffusion Loss (CDL), a self-supervised regularizer that trains a diffusion denoiser to classify noise levels, improving OOD denoising and parallel/sequential sampling quality.
Problem
Diffusion denoisers are poorly estimated in regions far outside the training distribution, which hurts sample quality especially for parallel (Picard-iteration) samplers that evaluate many OOD points.
Approach
Diffusion models implicitly define a log-likelihood ratio between noise levels, so the optimal denoiser is also an optimal noise classifier; adding a contrastive cross-entropy loss over noise-level pairs trains the denoiser on asymmetric (OOD) noise-level pairs.
Key results
- Parallel DDPM sampler FID on CIFAR-10 unconditional improves from 3.24 (VP) to 2.51 with CDL, and from 3.00 (VE) to 2.38 with CDL-VE.
- CDL-regularized models converge faster in parallel sampling on the 2D Dino dataset, with lower final MMD than the DDPM-loss baseline.
- With sequential EDM deterministic sampler, CDL is similar or better across datasets (e.g., FFHQ VP 2.38 to 2.29), and CDL-VE improves FFHQ FID from 2.56 to 2.47.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CDL to VP | VP · CIFAR-10 32x32 unconditional, parallel DDPM | FID: 3.24 ± 0.02 -> 2.51 ± 0.01 | Helped |
| Add CDL to VP | VP · CIFAR-10 32x32 conditional, parallel DDPM | FID: 2.93 ± 0.02 -> 2.41 ± 0.01 | Helped |
| Add CDL to VP | VP · AFHQv2 64x64 unconditional, parallel DDPM | FID: 2.95 ± 0.03 -> 2.91 ± 0.02 | Helped |
| Add CDL to VP | VP · FFHQ 64x64 unconditional, parallel DDPM | FID: 3.67 ± 0.04 -> 3.33 ± 0.03 | Helped |
| Add CDL to VE | VE · CIFAR-10 32x32 unconditional, parallel DDPM | FID: 3.00 ± 0.01 -> 2.38 ± 0.01 | Helped |
| Add CDL to VE | VE · CIFAR-10 32x32 conditional, parallel DDPM | FID: 2.76 ± 0.01 -> 2.25 ± 0.02 | Helped |
| Add CDL to VE | VE · AFHQv2 64x64 unconditional, parallel DDPM | FID: 2.98 ± 0.03 -> 2.93 ± 0.01 | Hurt |
| Add CDL to VE | VE · FFHQ 64x64 unconditional, parallel DDPM | FID: 3.65 ± 0.02 -> 3.29 ± 0.02 | Helped |
| Add CDL to VP | VP · CIFAR-10 32x32 unconditional, sequential deterministic EDM | FID: 2.00 ± 0.02 -> 1.99 ± 0.04 | No clear effect |
| Add CDL to VP | VP · AFHQv2 64x64 unconditional, sequential deterministic EDM | FID: 2.04 ± 0.00 -> 2.00 ± 0.00 | Helped |
| Add CDL to VE | VE · CIFAR-10 32x32 unconditional, sequential deterministic EDM | FID: 2.01 ± 0.01 -> 2.01 ± 0.01 | No clear effect |
| Add CDL to VE | VE · AFHQv2 64x64 unconditional, sequential deterministic EDM | FID: 2.17 ± 0.00 -> 2.11 ± 0.01 | Helped |
Surprising findings
- FID of the Karras deterministic EDM sampler deteriorates as NFE increases, and CDL-regularized models show a more stable FID across NFE.
- CDL-regularized stochastic samplers outperform EDM even at its optimal S_noise setting, suggesting CDL can remove the need to tune S_noise.
Limitations
- CDL training is substantially more expensive than standard diffusion loss (about 100x per point estimate), so only one data point per batch is used for the contrastive term.
- Experiments are limited to small-scale image datasets (CIFAR-10, FFHQ 64x64, AFHQv2 64x64) and a 2D synthetic dataset.
Experimental setup
Datasets: CIFAR-10 FFHQ 64x64 AFHQv2 64x64 2D Dino synthetic dataset
Benchmarks: CIFAR-10 FFHQ 64x64 AFHQv2 64x64 2D Dino synthetic
Metrics: FID MMD number of function evaluations Picard iterations wall-clock time
Baselines: DDPM EDM (VP/VE) Parallel DDPM sampler Karras deterministic/stochastic sampler
Builds on: DDPM EDM Parallel sampling with Picard iteration noise contrastive estimation
Closest papers by meaning
- Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.90)
- Test-Time Scaling of Diffusion Models via Noise Trajectory Search (NeurIPS 2025 · similarity 0.90)
- Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data (ICML 2024 · similarity 0.89)
- Training-Free Safe Denoisers for Safe Use of Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Score-Optimal Diffusion Schedules (NeurIPS 2024 · similarity 0.89)
- Ambient Diffusion Omni: Training Good Models with Bad Data (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.