Research map / Image generation / GANs
Improving Adversarial Energy-Based Model via Diffusion Process
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DDAEBM, which embeds adversarial energy-based models into each step of a diffusion process (T=4) so that each EBM learns a conditional denoising distribution, trained MCMC-free with a symmetric Jeffrey divergence and a variational latent posterior.
Problem
Adversarial energy-based models (EBMs) are MCMC-free and fast but suffer unstable minimax training, asymmetric KL-based generator objectives, and intractable entropy terms, leaving a generation-quality gap versus mainstream generative models.
Approach
Combine adversarial EBMs with a denoising diffusion process: at each noise step an energy function models a conditional denoising distribution, a latent-variable generator defines the implicit conditional, a symmetric Jeffrey divergence is used for the generator, and a variational posterior q_psi gives a tractable entropy bound.
Key results
- On CIFAR-10 the model reaches FID 4.82 and IS 8.86 with 4 NFE, the best among no-MCMC adversarial EBMs (e.g., BiDVL 20.75, VERA 27.5).
- On CelebA 64x64 it reaches FID 10.29, best among adversarial EBMs and better than NCSNv2 (26.86).
- Adding one extra denoising step with the energy function lowers CIFAR-10 FID to 3.73, on par with DDGAN (3.75).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a latent variable to the generated denoising distribution (ablation) | DDAEBM · CIFAR-10 | FID: 10.09 -> 4.82 | Helped |
| add a latent variable to the generated denoising distribution (ablation) | DDAEBM · CIFAR-10 | IS: 8.25 -> 8.86 | Helped |
| add a latent variable to the generated denoising distribution (ablation) | DDAEBM · OOD SVHN (CIFAR-10 in-distribution) | AUROC: 0.23 -> 0.83 | Helped |
| use symmetric Jeffrey divergence instead of KL divergence (ablation) | DDAEBM · CIFAR-10 | FID: 4.99 -> 4.82 | Helped |
| use symmetric Jeffrey divergence instead of KL divergence (ablation) | DDAEBM · CIFAR-10 | IS: 9.11 -> 8.86 | Hurt |
| use symmetric Jeffrey divergence instead of KL divergence (ablation) | DDAEBM · OOD SVHN (CIFAR-10 in-distribution) | AUROC: 0.37 -> 0.83 | Helped |
| include the qψ posterior-related objective term (ablation) | DDAEBM with KL divergence · CIFAR-10 | FID: 4.90 -> 4.99 | No clear effect |
| include the qψ posterior-related objective term (ablation) | DDAEBM with KL divergence · CIFAR-10 | IS: 8.90 -> 9.11 | No clear effect |
| include the qψ posterior-related objective term (ablation) | DDAEBM with KL divergence · OOD SVHN (CIFAR-10 in-distribution) | AUROC: 0.38 -> 0.37 | No clear effect |
| add an additional denoising step using the energy function (ablation) | DDAEBM · CIFAR-10 | FID: 4.82 -> 3.73 | Helped |
| increase the number of diffusion time steps from 1 to 4 (ablation) | DDAEBM · CIFAR-10 | FID | Helped |
| increase the number of diffusion time steps beyond 4 (ablation) | DDAEBM · CIFAR-10 | FID | Hurt |
Surprising findings
- The model is still less performant than DDGAN despite sharing the same generator structure, which the authors attribute to the energy function carrying a heavier density-estimation workload.
- On MNIST NICE-based likelihood, the method underperforms WGAN-based baselines (-902 vs. -1021 for VERA is actually better; DDAEBM is best among the listed non-MLE methods) while on OOD detection it is only moderate on CIFAR-100 and CelebA.
Limitations
- Performance lags DDGAN on CIFAR-10 without an extra refinement step.
- Larger numbers of time steps degrade performance, so T must be tuned (T=4 chosen).
- OOD detection is mixed, with no clear winner across datasets; CIFAR-100 and CelebA results are moderate.
Experimental setup
Datasets: 25-Gaussians pinwheel Swiss roll MNIST CIFAR-10 CelebA LSUN church
Benchmarks: CIFAR-10 CelebA 64x64 LSUN church 128x128 MNIST 2D toy datasets CIFAR-10/SVHN OOD
Metrics: FID IS NFE test log-likelihood AUROC
Baselines: VERA MEG BiDVL FCE EBM Triangle EBM-BB WGAN-0GP DDGAN DRL VAEBM NCSNv2 DDIM
Builds on: DDGAN DRL BiDVL VERA DDPM Score SDE
Closest papers by meaning
- Training Deep Energy-Based Models with f-Divergence Minimization (ICML 2020 · similarity 0.91)
- Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood (ICLR 2024 · similarity 0.90)
- Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding (ICML 2024 · similarity 0.88)
- Improved Diffusion-based Generative Model with Better Adversarial Robustness (ICLR 2025 · similarity 0.87)
- Energy Matching: Unifying Flow Matching and Energy-Based Models for Generative Modeling (NeurIPS 2025 · similarity 0.87)
- Learning Latent Space Hierarchical EBM Diffusion Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.