Research map / Probabilistic and scientific modeling / Variational and generative sampling
Optimizing Unnormalized Statistical Models Through Compositional Optimization
Wei Jiang, Jiayu Qin, Lingyu Wu, Changyou Chen, Tianbao Yang, Lijun Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| optimize NLL directly via compositional optimization instead of NCE | unnormalized model · Synthetic 2spirals (SGD) | MMD: 3.253 -> 3.040 | Improved |
| replace MCMC training with compositional method | EBM training · 1-d Gaussian running time | running time: 1757s -> 168s | Improved |
| use compositional objective instead of NCE on MNIST EBM | ResNet-18 EBM · MNIST test | NLL (bits/dim): 1.457 -> 1.394 | Improved |
| replace JEM Langevin EBM training with compositional method | JEM · CIFAR-10 classification | accuracy: 89.38% -> 90.72% | Improved |
| replace JEM Langevin EBM training with compositional method | JEM · CIFAR-10 generation | FID: 54.76 -> 51.77 | Improved |
| replace JEM Langevin EBM training with compositional method | JEM · CIFAR-10 generation | Inception Score: 4.83 -> 4.79 | No clear effect |
| use mixture of empirical and Gaussian noise instead of either alone (ablation) | Ours · MNIST generation | sample quality | Improved |
| increase beta from 0.8 to 0.9 (ablation) | Ours · Synthetic 2spirals (SGD) | MMD: 3.355 -> 3.040 | Improved |
| increase beta from 0.9 to 0.99 (ablation) | Ours · Synthetic 2spirals (SGD) | MMD: 3.040 -> 3.038 | No clear effect |
| increase gamma from 0.01 to 0.1 (ablation) | Ours · Synthetic circles (SGD) | MMD: 1.345 -> 1.075 | Improved |
| increase gamma from 0.1 to 0.2 (ablation) | Ours · Synthetic 2spirals (SGD) | MMD: 3.040 -> 3.168 | Hurt |
| train NCE with natural gradient descent | NCE · Synthetic 2spirals (SGD) | MMD: 3.253 -> 3.445 | Hurt |
Closest papers by meaning
- Learning Unnormalized Statistical Models via Compositional Optimization (ICML 2023 · similarity 0.99)
- A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation (ICML 2025 · similarity 0.89)
- Near-Optimality of Contrastive Divergence Algorithms (NeurIPS 2024 · similarity 0.86)
- Information Theoretic Learning for Diffusion Models with Warm Start (NeurIPS 2025 · similarity 0.84)
- Learning normalized image densities via dual score matching (NeurIPS 2025 · similarity 0.84)
- Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
