Research map / Image generation / Autoregressive and latent generation
Improved Contrastive Divergence Training of Energy-Based Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add data augmentation transitions to Langevin sampling | Energy-based model sampling · Image sampling and mode mixing | chain diversity | Helped |
| Add multiscale sampling (ablation) | CIFAR-10 EBM with data augmentation · CIFAR-10 unconditional generation | FID: 133.84 -> 53.78 | Helped |
| Add data augmentation (ablation) | CIFAR-10 EBM without KL loss or multiscale sampling · CIFAR-10 unconditional generation | Inception: 3.57 -> 5.13 | Helped |
| Add the KL energy-minimization loss (ablation) | CIFAR-10 EBM with data augmentation and multiscale sampling · CIFAR-10 unconditional generation | FID: 53.78 -> 32.67 | Helped |
| Add the entropy component of the KL loss (ablation) | CIFAR-10 EBM with energy-minimization KL loss, data augmentation, and multiscale sampling · CIFAR-10 unconditional generation | FID: 32.67 -> 25.08 | Helped |
| Add the KL loss term | EBM training · CIFAR-10 training stability | training stability | Helped |
| Use spectral normalization instead of the KL loss | EBM with self-attention · EBM training stability | training stability | Hurt |
| Truncate backpropagation through Langevin sampling to the last step (ablation) | MNIST EBM · MNIST generation | FID: 94.85 -> 90.54 | No clear effect |
| Backpropagate through all Langevin steps rather than only the last step (ablation) | MNIST EBM · MNIST training speed | training time: over 3 times slower | Hurt |
| Use the proposed EBM framework | CIFAR-10 unconditional model · SVHN OOD detection | AUROC: 0.83 -> 0.91 | Helped |
| Use the proposed EBM framework | CIFAR-10 unconditional model · CIFAR-10 interpolations OOD detection | AUROC: 0.86 -> 0.65 | Hurt |
| Use the proposed EBM framework | CIFAR-10 unconditional model · CIFAR-100 OOD detection | AUROC: 0.73 -> 0.83 | Helped |
Closest papers by meaning
- Near-Optimality of Contrastive Divergence Algorithms (NeurIPS 2024 · similarity 0.88)
- Simple Distillation for One-Step Diffusion Models (NeurIPS 2025 · similarity 0.87)
- Improving Adversarial Energy-Based Model via Diffusion Process (ICML 2024 · similarity 0.87)
- Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood (ICLR 2024 · similarity 0.86)
- Equilibrium Matching: Generative Modeling with Implicit Energy-Based Models (NeurIPS 2026 · similarity 0.86)
- Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.