Research map / Optimization / Distributed and compressed
Stochastic Gradient MCMC with Stale Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use stale gradients computed asynchronously by multiple workers instead of current gradients | S²G-MCMC · SG-MCMC posterior estimation | estimation variance | Helped |
| Increase the number of workers in the single-server architecture | S²G-MCMC · Adult a9a | estimation variance | Helped |
| Increase the number of workers in the single-server architecture | S²G-MCMC · MNIST | test loss convergence | Helped |
| Increase the number of workers beyond 4 | S²G-MCMC · CIFAR-10 | final test loss | Hurt |
| Increase the number of servers | S²G-MCMC · Bayesian logistic regression (a9a) | test loss convergence | No clear effect |
| Increase the number of servers | S²G-MCMC · MNIST | test loss convergence | Helped |
| Increase the number of servers | S²G-MCMC · CIFAR-10 | test loss convergence | Helped |
| Increase gradient staleness while setting iterations to L = L₀ × τ | Distributed SGLD · Synthetic Gaussian model | MSE | No clear effect |
| Use multiple independent chains | S²G-MCMC · Posterior estimation | bias | No clear effect |
| Use multiple independent chains | S²G-MCMC · Posterior estimation | MSE | Helped |
Closest papers by meaning
- Communication-Efficient Stochastic Gradient MCMC for Neural Networks (AAAI 2019 · similarity 0.91)
- High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models (AAAI 2016 · similarity 0.90)
- Structured Stochastic Gradient MCMC (ICML 2022 · similarity 0.89)
- Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning (ICLR 2020 · similarity 0.89)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.88)
- Learning to Explore for Stochastic Gradient MCMC (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.