Research map / Optimization / Optimizers
High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the Euler integrator with the symmetric splitting integrator (ablation) | mSGNHT · ICML LDA test | test perplexity: 960.56 -> 939.67 | Helped |
| Replace the Euler integrator with the symmetric splitting integrator (ablation) | mSGNHT · a9a logistic regression test | test accuracy: 84.72% -> 84.95% | Helped |
| Replace the Euler integrator with the symmetric splitting integrator (ablation) | mSGNHT · MNIST 400-400 ReLU test | accuracy: 98.20 -> 98.25 | Helped |
| Replace the Euler integrator with the symmetric splitting integrator (ablation) | mSGNHT · MNIST 3-layer sigmoid FNN test | accuracy: 96.86% -> 97.36% | Helped |
| Replace the Euler integrator with the symmetric splitting integrator (ablation) | mSGNHT · MNIST deep FNN | test accuracy | Helped |
| Use the symmetric splitting integrator rather than Euler (ablation) | mSGNHT · Wikipedia DPFA | predictive perplexity | Helped |
| Use the symmetric splitting integrator rather than Euler (ablation) | mSGNHT · Double-well potential | KL divergence | Helped |
| Use a larger stepsize (ablation) | mSGNHT-E · Double-well potential | parameter distribution accuracy | Hurt |
| Decrease the stepsize by half during training (ablation) | mSGNHT-E · MNIST 4-layer FNN | training stability | Helped |
| Use a larger stepsize (ablation) | mSGNHT-S · MNIST 2-layer FNN | convergence speed | Helped |
| Replace mSGNHT with SGHMC | Bayesian sampling · ICML LDA test | test perplexity: 939.67 -> 1004.73 | Hurt |
| Replace mSGNHT-S with Gibbs sampling | Bayesian sampling · ICML LDA test | test perplexity: 939.67 -> 907.84 | Helped |
Closest papers by meaning
- Meta-Learning For Stochastic Gradient MCMC (ICLR 2019 · similarity 0.92)
- Structured Stochastic Gradient MCMC (ICML 2022 · similarity 0.91)
- Stochastic Gradient MCMC with Stale Gradients (NeurIPS 2016 · similarity 0.90)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.89)
- Learning to Explore for Stochastic Gradient MCMC (ICML 2024 · similarity 0.89)
- Entropy-MCMC: Sampling from Flat Basins with Ease (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.