Research map / Probabilistic and scientific modeling / Variational and generative sampling
Marginalized Stochastic Natural Gradients for Black-Box Variational Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Marginalizing discrete latent variables and using natural gradients (MSNG) instead of REINFORCE | Black-box VI on discrete latent variable models · Eight model-dataset combinations (Newsgroups, MNIST, Countries, NeurIPS, RTE, WSD) | convergence speed (runtime and iterations): tens to hundreds of times faster | Helped |
| Adding control variates to REINFORCE | REINFORCE · Annotation and larger NeurIPS relational models | ELBO improvement speed | Helped |
| Using natural gradients (SNG) instead of plain REINFORCE gradients | Stochastic gradient VI without marginalization · Simple problems (deep belief networks, small Countries relational data) | convergence speed | Helped |
| Using MSNG versus SCAVI (heuristic stochastic CAVI) | Variational inference with discrete latents · Sigmoid belief net, relational and annotation models | final ELBO and stability | Helped |
| CONCRETE (Gumbel-based biased relaxation) instead of MSNG | Discrete latent variable models · Various models | final ELBO | Hurt |
| Using auxiliary-variable model-specific bounds (AUX) instead of MSNG | Binary models (noisy-OR, sigmoid, probit) · MNIST sigmoid belief network and relational models | final ELBO | Hurt |
Closest papers by meaning
- Robust, Accurate Stochastic Optimization for Variational Inference (NeurIPS 2020 · similarity 0.93)
- Understanding Stochastic Natural Gradient Variational Inference (ICML 2024 · similarity 0.91)
- Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference (NeurIPS 2025 · similarity 0.90)
- Natural Gradient VI: Guarantees for Non-Conjugate Models (NeurIPS 2025 · similarity 0.90)
- Globally Convergent Variational Inference (NeurIPS 2024 · similarity 0.90)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.