Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Distributed Bayesian Learning with Stochastic Natural Gradient Expectation Propagation and the Posterior Server
Leonard Hasenclever, Stefan Webb, Thibaut Lienart, Sebastian Vollmer, Balaji Lakshminarayanan, Charles Blundell, Yee Whye Teh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| set power parameter beta=1/Nworkers (p-SNEP) instead of beta=1 (ablation) | SNEP · MNIST 2-layer FC | convergence speed: eight workers reach a test error of 1.5% after about two epochs, much faster than beta=1.0 | Improved |
| add a second worker (vs one worker or Adam) (ablation) | SNEP (beta=1) · MNIST 2-layer FC | test error | Improved |
| increase workers beyond two (ablation) | SNEP (beta=1) · MNIST 2-layer FC | test error | Hurt |
| increase workers with beta=1/Nworkers (ablation) | p-SNEP · MNIST 2-layer FC | test error and convergence | Improved |
| communicate less often (Nsync up to once every 100 iterations) (ablation) | SNEP · MNIST 2-layer FC | test error | No clear effect |
| lower beta (ablation) | SNEP · MNIST 2-layer FC | convergence speed | Improved |
| use SNEP instead of SMS under Monte Carlo noise | distributed logistic regression (N=50000, d=50) · synthetic logistic regression | predictive RMSE | Improved |
| use p-SNEP instead of A-SGD and EASGD | 20x50 deep FC network · MNIST deep FC (8 and 16 workers) | accuracy: an extra percent of accuracy after fifty epochs | Improved |
| add noise to SGD methods (ablation) | deep FC MNIST · MNIST deep FC | run variance | Improved |
| add L2 regularization (ablation) | A-SGD and EASGD · MNIST deep FC | accuracy | No clear effect |
| use SNEP (beta=1) instead of p-SNEP | CIFAR-10 CNN · CIFAR-10 | final test error | Improved |
Closest papers by meaning
- Expectation Propagation as a Way of Life: A Framework for Bayesian Inference on Partitioned Data (JMLR 2020 · similarity 0.89)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.88)
- Structured Stochastic Gradient MCMC (ICML 2022 · similarity 0.87)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.86)
- Scalable Bayesian Learning with posteriors (ICLR 2025 · similarity 0.86)
- Fearless Stochasticity in Expectation Propagation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
