Research map / Optimization / Optimizers
Non-convex Learning via Replica Exchange Stochastic Gradient MCMC
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive replica exchange SGHMC instead of SGHMC | Bayesian GAN · CIFAR10 semi-supervised, 2000–5000 labels | accuracy: 74.72$\pm$0.39 -> 77.73$\pm$0.31; 77.96$\pm$0.32 -> 80.85$\pm$0.23; 79.06$\pm$0.29 -> 81.61$\pm$0.24; 81.74$\pm$0.36 -> | Helped |
| Use adaptive replica exchange SGHMC instead of SGHMC | Bayesian GAN · CIFAR100 semi-supervised, 2000–5000 labels | accuracy: 50.76$\pm$0.71 -> 55.53$\pm$ 0.64; 53.07$\pm$0.71 -> 57.09$\pm$ 0.77; 57.05$\pm$0.59 -> 62.23$\pm$ 0.69; 59.34$\pm$0.64 | Helped |
| Use adaptive replica exchange SGHMC instead of SGHMC | Bayesian GAN · SVHN semi-supervised, 2000–3000 labels | accuracy: 88.75$\pm$0.44 -> 91.59$\pm$0.38; 91.32$\pm$0.41 -> 94.03$\pm$0.36 | Helped |
| Use reSGHMC rather than SGHMC | ResNet-20, ResNet-32, ResNet-56, WRN-16-8, and WRN-28-10 · CIFAR10 test | accuracy: 94.22$\pm$0.12 -> 94.62$\pm$0.18; 95.18$\pm$0.06 -> 95.35$\pm$0.08; 95.95$\pm$0.10 -> 96.12$\pm$0.06; 96.73$\pm$0.08 -> | Helped |
| Use reSGHMC rather than SGHMC | ResNet-20, ResNet-32, ResNet-56, WRN-16-8, and WRN-28-10 · CIFAR100 test | accuracy: 72.49$\pm$0.18 -> 74.14$\pm$0.22; 75.14$\pm$0.28 -> 76.55$\pm$0.30; 79.04$\pm$0.30 -> 80.14$\pm$0.34; 82.07$\pm$0.22 -> | Helped |
| Increase batch size from 256 to 1024 (ablation) | ResNet-20 reSGHMC · CIFAR10 test | accuracy: 94.62$\pm$0.18 -> 95.01$\pm$0.16 | Helped |
| Increase batch size from 256 to 1024 (ablation) | ResNet-20 reSGHMC · CIFAR100 test | accuracy: 74.14$\pm$0.21 -> 75.11$\pm$0.26 | Helped |
| Increase correction factor F from 2 to 4 or infinity (ablation) | reSGLD on the third Gaussian mixture with heavy-tailed energy noise · Gaussian mixture U3 | approximation error: 0.4%; 7.1%; 46% | Hurt |
| Use correction factor F=2 rather than F=1 (ablation) | reSGLD on the third Gaussian mixture with heavy-tailed energy noise · Gaussian mixture U3 | approximation to ground truth: 0.4% | Helped |
| Remove the swap correction (ablation) | reSGLD on the first Gaussian mixture · Gaussian mixture U1 | distributional approximation | Hurt |
| Use adaptive variance correction under heavy-tailed energy noise rather than naive reSGLD | reSGLD on the second Gaussian mixture · Gaussian mixture U2 | approximation to ground truth | Helped |
Closest papers by meaning
- Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction (ICLR 2021 · similarity 0.91)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.87)
- Constrained Exploration via Reflected Replica Exchange Stochastic Gradient Langevin Dynamics (ICML 2024 · similarity 0.87)
- Parallelizing MCMC Across the Sequence Length (NeurIPS 2025 · similarity 0.86)
- AutoStep: Locally adaptive involutive MCMC (ICML 2025 · similarity 0.86)
- Reverse Diffusion Monte Carlo (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.