Research map / Optimization / Optimizers
Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace decreasing stepsizes with a cyclical schedule | SG-MCMC on CIFAR-10 and CIFAR-100 · CIFAR-10 and CIFAR-100 | test error (%): 5.20±0.06 -> 4.29±0.06; 23.23±0.01 -> 20.55±0.06 | Helped |
| replace decreasing stepsizes with a cyclical schedule | SGHMC on CIFAR-10 and CIFAR-100 · CIFAR-10 and CIFAR-100 | test error (%): 4.93±0.1 -> 4.27±0.03; 22.60±0.17 -> 20.50±0.11 | Helped |
| use cyclical rather than decreasing stepsizes | SGHMC on ImageNet · ImageNet testing set | NLL: 0.9308 -> 0.8882 | Helped |
| use cyclical rather than decreasing stepsizes | SGLD on a 25-Gaussian mixture · 25-Gaussian mixture, non-parallel | mode coverage: 1.8±0.13 -> 6.7±0.52 | Helped |
| use cyclical rather than decreasing stepsizes | parallel SGLD on a 25-Gaussian mixture · 25-Gaussian mixture, parallel | mode coverage: 18±0.47 -> 24.4±0.22 | Helped |
| use cyclical rather than decreasing stepsizes | SGHMC for Bayesian logistic regression · Australian | effective sample size: 1317 -> 4707 | Helped |
| use cyclical rather than decreasing stepsizes | SGHMC for Bayesian logistic regression · German | effective sample size: 2007 -> 2436 | Helped |
| use cyclical rather than decreasing stepsizes | SGHMC for Bayesian logistic regression · Heart | effective sample size: 5000 -> 5000 | No clear effect |
| use cyclical rather than decreasing stepsizes in parallel chains | SGLD · CIFAR-10 and CIFAR-100 | test error (%): 4.15 -> 4.09; 20.29 -> 19.37 | Helped |
| use cyclical rather than decreasing stepsizes in parallel chains | SGHMC · CIFAR-10 and CIFAR-100 | test error (%): 4.09 -> 3.95; 19.72 -> 19.19 | Helped |
| use cyclical single-chain sampling instead of decreasing parallel sampling | SGLD · CIFAR-10 and CIFAR-100 | test error (%): 5.11 -> 4.29; 21.16 -> 20.55 | Mixed |
| use cyclical single-chain sampling instead of decreasing parallel sampling | SGHMC · CIFAR-10 and CIFAR-100 | test error (%): 4.52 -> 4.27; 20.82 -> 20.50 | Mixed |
Closest papers by meaning
- Learning to Explore for Stochastic Gradient MCMC (ICML 2024 · similarity 0.91)
- High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models (AAAI 2016 · similarity 0.89)
- Stochastic Gradient MCMC with Stale Gradients (NeurIPS 2016 · similarity 0.89)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.87)
- Gradient-based Discrete Sampling with Automatic Cyclical Scheduling (NeurIPS 2024 · similarity 0.86)
- A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.