Research map / Probabilistic and scientific modeling / Variational and generative sampling
The Variational Predictive Natural Gradient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VPNG instead of vanilla gradient optimization | Mean-field Bayesian Logistic regression · Synthetic Bayesian Logistic regression train | AUC: 0.734\pm0.017 -> \mb{0.972\pm0.011} | Helped |
| Use VPNG instead of traditional natural gradient optimization | Mean-field Bayesian Logistic regression · Synthetic Bayesian Logistic regression train | AUC: 0.744\pm0.043 -> \mb{0.972\pm0.011} | Helped |
| Use VPNG instead of vanilla gradient optimization | Mean-field Bayesian Logistic regression · Synthetic Bayesian Logistic regression test | AUC: 0.718\pm0.022 -> \mb{0.967\pm0.011} | Helped |
| Use VPNG instead of traditional natural gradient optimization | Mean-field Bayesian Logistic regression · Synthetic Bayesian Logistic regression test | AUC: 0.751\pm0.047 -> \mb{0.967\pm0.011} | Helped |
| Use VPNG instead of baseline gradient optimizations | VAE · Binarized MNIST train and test | ELBO | Helped |
| Use VPNG instead of baseline gradient optimizations | Variational matrix factorization · MovieLens 20M train and test | ELBO | Helped |
| Remove exponential moving averages from the curvature estimates (ablation) | VPNG for VAE · Binarized MNIST | ELBO optimization performance | No clear effect |
| Remove exponential moving averages from the curvature estimates (ablation) | Traditional natural gradient for VAE · Binarized MNIST | optimization stability and efficiency | Hurt |
| Vary the dampening factor, EMA decay, or low-rank approximation parameter (ablation) | VPNG and natural gradient · VAE and variational matrix factorization | performance | No clear effect |
Closest papers by meaning
- Marginalized Stochastic Natural Gradients for Black-Box Variational Inference (ICML 2021 · similarity 0.91)
- Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference (NeurIPS 2017 · similarity 0.90)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.90)
- Globally Convergent Variational Inference (NeurIPS 2024 · similarity 0.89)
- Natural Gradient VI: Guarantees for Non-Conjugate Models (NeurIPS 2025 · similarity 0.88)
- Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.