Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Noisy Natural Gradient as Variational Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a matrix-variate Gaussian posterior rather than a fully factorized posterior (ablation) | NNG · UCI regression benchmarks | Test RMSE and test log-likelihood: Boston RMSE: 3.031$\pm$0.155 -> 2.742$\pm$0.125; Kin8nm RMSE: 0.079$\pm$0.001 -> 0.076$\pm$0.001 | Helped |
| use a fully factorized Gaussian posterior instead of matrix-variate Gaussian (ablation) | NNG · UCI regression benchmarks | Test RMSE and test log-likelihood: Wine RMSE: 0.644$\pm$0.011 vs 0.637$\pm$0.011; Naval log-likelihood: 6.231$\pm$0.041 vs 7.079$\pm$0.034 | Mixed |
| replace K-FAC with Noisy-KFAC adaptive weight noise | CIFAR10 VGG16 · CIFAR10 without additional augmentation or batch normalization | Test accuracy: 82.39 -> 85.52 | Helped |
| add matched-magnitude weight decay to K-FAC (ablation) | K-FAC · CIFAR10 VGG16 | Test accuracy: K-FAC with this weight decay setting achieves 83.51%; noisy K-FAC achieves 85.52% | Helped |
| use Noisy-KFAC rather than K-FAC | CIFAR10 VGG16 with data augmentation and batch normalization · CIFAR10 | Test accuracy: 92.13 -> 92.01 | Hurt |
| use NNG-MVG instead of NNG-FFG (ablation) | NNG · Predictive variance correlation with HMC across UCI datasets | Pearson correlation: Boston: 0.718$\pm$0.035 -> 0.891$\pm$0.021; Kin8nm: 0.659$\pm$0.015 -> 0.731$\pm$0.021 | Helped |
| use NNG-BlkTri instead of NNG-MVG (ablation) | NNG · Predictive variance correlation with HMC across UCI datasets | Pearson correlation: Boston: 0.891$\pm$0.021 -> 0.889$\pm$0.024; Concrete: 0.913$\pm$0.010 -> 0.922$\pm$0.006 | Helped |
| use NNG-MVG for active learning rather than random selection (ablation) | NNG-MVG · UCI active learning | Test RMSE: Boston: 5.831$\pm$0.177 -> 5.220$\pm$0.132; Kin8nm: 0.243$\pm$0.003 -> 0.244$\pm$0.003 | Mixed |
| replace BBB dynamics model with NNG-MVG | VIME · CartPoleSwingup, MountainCar, and DoublePendulum | Average return | Helped |
| use noisy Adam instead of BBB | Fully factorized Gaussian variational BNN · UCI regression benchmarks | ELBO convergence | Helped |
Closest papers by meaning
- Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam (ICML 2018 · similarity 0.92)
- The Variational Predictive Natural Gradient (ICML 2019 · similarity 0.90)
- Convergence Rates of Variational Inference in Sparse Deep Learning (ICML 2020 · similarity 0.90)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.89)
- Globally Convergent Variational Inference (NeurIPS 2024 · similarity 0.89)
- VIKING: Deep variational inference with stochastic projections (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.