Research map / Optimization / Optimizers
Learning Unnormalized Statistical Models via Compositional Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace NCE objective with MECO direct MLE via compositional optimization | unnormalized models on synthetic 2D density estimation · 6 synthetic 2D datasets (MMD) | MMD (lower better) | Helped |
| Use MECO (compositional MLE) instead of standard EBM/CD training on CIFAR-10 JEM codebase | JEM · CIFAR-10 generation FID | FID (lower better): 54.76 -> 51.77 | Helped |
| Use MECO instead of Langevin EBM training on JEM | JEM · CIFAR-10 classification accuracy | accuracy: 89.38% -> 90.72% | Helped |
| Use empirical-data + noise mixture as noise distribution for MECO on CIFAR-10 JEM (ablation) | MECO on CIFAR-10 · CIFAR-10 generation | sample quality | Hurt |
| Use mixture of empirical data and fitted Gaussian as noise (vs pure empirical or pure Gaussian) (ablation) | MECO EBM on MNIST · MNIST generation quality | visual sample quality | Helped |
| Increase noise mini-batch size for estimating g and grad g | MECO · theory | variance impact on convergence | Helped |
| Use MECO with Adam optimizer instead of SGD on synthetic data | MECO · Synthetic 2D (MMD) | MMD (lower better) | No clear effect |
| Set beta to 0.8 instead of 0.9 (ablation) | MECO · Synthetic 2D 2spirals MMD | MMD (lower better): 3.040 -> 3.355 | Hurt |
| Set gamma to 0.01 or 0.2 instead of 0.1 (ablation) | MECO · Synthetic 2D 2spirals MMD | MMD (lower better): 2.982 and 3.168 vs 3.040 | No clear effect |
| Use MECO (MLE) instead of NCE on OOD detection with CIFAR-10 as in-distribution | energy-based model (WideResNet-40) · OOD AUROC across CIFAR10-Interp, SVHN, CIFAR-100, LSUN-C | AUROC (higher better) | Helped |
| Use MECO instead of NCE, eNCE, and MCMC on Gaussian mean estimation runtime-matched comparison | MECO · 1-d Gaussian mean estimation | running time: 168s vs NCE 176s | No clear effect |
Closest papers by meaning
- A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation (ICML 2025 · similarity 0.90)
- Near-Optimality of Contrastive Divergence Algorithms (NeurIPS 2024 · similarity 0.86)
- Learning normalized image densities via dual score matching (NeurIPS 2025 · similarity 0.85)
- Information Theoretic Learning for Diffusion Models with Warm Start (NeurIPS 2025 · similarity 0.85)
- Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences (NeurIPS 2026 · similarity 0.85)
- Sampling Binary Data by Denoising through Score Functions (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.