Research map / Optimization / Optimizers
Communication-Efficient Stochastic Gradient MCMC for Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use P=4 workers with pi=3 instead of single worker | Downpour pSGLD · MNIST FNN 400-400 | wallclock time (s): 3188 -> 2664 | Helped |
| use P=4 workers with pi=3 instead of single worker | Downpour pSGLD · MNIST FNN 400-400 | test error: 1.40% -> 1.34% | No clear effect |
| use P=4 workers with pi=10 instead of single worker | Elastic SGHMC · MNIST FNN 1200-1200 | wallclock time (s): 6776 -> 5552 | Helped |
| use P=4 workers with pi=10 instead of single worker | Elastic SGHMC · MNIST FNN 400-400 | test error: 1.76% -> 1.79% | No clear effect |
| increase communication period pi with P=4 workers (ablation) | Downpour SGLD · MNIST FNN 400-400 | test NLL vs time | Hurt |
| increase communication period pi with P=4 workers (ablation) | Elastic SGHMC · MNIST FNN 400-400 | test NLL vs time | No clear effect |
| increase workers P beyond a limit (ablation) | Downpour SGLD · MNIST FNN 400-400 | learning speed | Hurt |
| use SG-MCMC instead of distributed optimization (RMSprop, AMSGD) | downpour/elastic protocols · MNIST FNN | test NLL overfitting | Helped |
| use pSGLD sampling instead of RMSprop in A3C | A3C · Cartpole-v1 | reward | Helped |
| use 4 workers instead of 1 on GRU | Downpour pSGLD / Elastic SGHMC · War and Peace char LM, 2-layer GRU | test NLL | Helped |
Closest papers by meaning
- Stochastic Gradient MCMC with Stale Gradients (NeurIPS 2016 · similarity 0.91)
- Structured Stochastic Gradient MCMC (ICML 2022 · similarity 0.90)
- Stochastic Gradient MCMC Methods for Hidden Markov Models (ICML 2017 · similarity 0.90)
- High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models (AAAI 2016 · similarity 0.89)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.88)
- Learning to Explore for Stochastic Gradient MCMC (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.