Research map / Optimization / Optimizers
Constrained Exploration via Reflected Replica Exchange Stochastic Gradient Langevin Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes reflected reSGLD (r2SGLD), a replica-exchange SGLD sampler that reflects out-of-domain samples back into a bounded domain, with theory on mixing rates vs. domain diameter and experiments on Lorenz/Lotka-Volterra identification, constrained multimodal sampling, and CIFAR-100 ResNets.
Problem
Replica exchange SGLD over-explores distribution tails at high temperature, causing stagnation and unstable training in non-convex sampling and optimization.
Approach
Restrict replica-exchange Langevin chains to a bounded domain using reflection steps (tangent-line symmetric reflection at the nearest boundary), with swaps between temperatures, and show mixing rate improves quadratically as domain diameter shrinks.
Key results
- On CIFAR-100 ResNet56, r2SGHMC reaches 79.39±0.30% accuracy with NLL 7155±91, versus 78.87±0.44% / 7406±130 for reSGHMC.
- On CIFAR-100 ResNet20, r2SGHMC reaches 75.38±0.17% accuracy and Brier 2.46‰, versus 75.01±0.14% for reSGHMC.
- Theory gives continuous-time chi^2 and W2 exponential convergence with a spectral-gap lower bound scaling as 1/diam(Omega)^2; experiments show smaller domain diameter yields faster KL decay.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add reflection to replica-exchange SGHMC | reSGHMC on ResNet20 · CIFAR100 ResNet20 | accuracy (ACC, %): 75.01\pm0.14 -> 75.38\pm0.17 | Helped |
| Add reflection to replica-exchange SGHMC | reSGHMC on ResNet20 · CIFAR100 ResNet20 | NLL: 8552\pm 69 -> 8489\pm66 | Helped |
| Add reflection to replica-exchange SGHMC | reSGHMC on ResNet20 · CIFAR100 ResNet20 | Brier (‰): 2.50\pm0.01 -> 2.46\pm0.02 | Helped |
| Add reflection to replica-exchange SGHMC | reSGHMC on ResNet56 · CIFAR100 ResNet56 | accuracy (ACC, %): 78.87\pm0.44 -> 79.39\pm0.30 | Helped |
| Add reflection to replica-exchange SGHMC | reSGHMC on ResNet56 · CIFAR100 ResNet56 | NLL: 7406\pm130 -> 7155\pm91 | Helped |
| Add reflection to replica-exchange SGHMC | reSGHMC on ResNet56 · CIFAR100 ResNet56 | Brier (‰): 2.94\pm0.06 -> 2.89\pm0.02 | Helped |
| Reduce the constrained-domain diameter | r2SGLD · Bounded multi-modal distributions | KL divergence | Helped |
| Use reflection instead of no reflection | Image-classification sampling methods · CIFAR100 | test accuracy | Helped |
| Use reflection instead of no reflection | Sampling methods for constrained multi-modal distributions · 2D mixture of 25 Gaussians in a flower-shaped boundary | KL divergence | Helped |
| Use dual-chain replica exchange rather than a single reflected chain | Reflected sampling methods · 2D mixture of 25 Gaussians in a flower-shaped boundary | KL divergence | Helped |
| Use a cyclical learning rate rather than a fixed learning rate | Reflected SGLD · 2D mixture of 25 Gaussians in a flower-shaped boundary | KL divergence | Helped |
| Use reflection rather than L2 penalty-based constraint handling | Constrained sampling methods · 2D mixture of 25 Gaussians in a flower-shaped boundary | KL divergence | Helped |
Surprising findings
- Algorithms without reflection often fail (test accuracy under 10% or infinite loss) at initial learning rates above 10.0, while reflection within [-4, 4] stabilizes training and gives over 70% accuracy.
Limitations
- Learning rates limited to about 2.0 for CIFAR-100 due to computational resources, below the 10+ where reflection shows its main benefit; runs limited to 1,000 epochs.
- Theoretical acceleration and spectral-gap bounds for non-convex domains are less explicit; the paper defers general non-convexity analysis to future work.
- Discretization analysis is only W1 and under convex domains.
Experimental setup
Datasets: CIFAR-100 Lorenz system trajectories (generated via Runge-Kutta) Lotka-Volterra trajectories (generated via LSODA) 2D 25-Gaussian mixture with constrained domains
Benchmarks: CIFAR-100 Lorenz system identification Lotka-Volterra system identification 2D 25-Gaussian mixture in constrained domains
Metrics: accuracy (BMA) NLL Brier score KL divergence posterior mode accuracy
Baselines: SGLD R-SGLD cycSGLD R-cycSGLD reSGLD P-SGLD SGDM SGHMC reSGHMC
Builds on: reSGLD replica exchange Langevin diffusion reflected SGLD cyclical SGLD SGHMC
Closest papers by meaning
- Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction (ICLR 2021 · similarity 0.91)
- Fast Non-Log-Concave Sampling under Nonconvex Equality and Inequality Constraints with Landing (NeurIPS 2025 · similarity 0.88)
- Langevin-Informed Transfer Learning: Replacing the Target Samples by Black-Box Feedback (NeurIPS 2026 · similarity 0.87)
- Generalization of noisy SGD in unbounded non-convex settings (ICML 2025 · similarity 0.87)
- NETS: A Non-equilibrium Transport Sampler (ICML 2025 · similarity 0.87)
- Constrained Sampling with Primal-Dual Langevin Monte Carlo (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.