Research map / Probabilistic and scientific modeling / Variational and generative sampling
Practical Hamiltonian Monte Carlo on Riemannian Manifolds via Relativity Theory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generalizes relativistic (velocity-bounded) HMC of Lu et al. (2017) to Riemannian manifolds with a position-dependent metric (GR-HMC), using a Box-Muller-based momentum sampler and diagonal SoftAbs Hessian approximations to improve integration stability.
Problem
Riemannian HMC with position-dependent metrics suffers from numerical integration instability (divergent transitions) in high-curvature regions, forcing very small step sizes.
Approach
Position-dependent relativistic kinetic energy that upper-bounds particle speed in a metric-dependent way, so velocity limits shrink in high-curvature regions; plus a Box-Muller-based multivariate sampler for the relativistic momentum that avoids the mean-field (dimension-wise) assumption.
Key results
- On Neal's funnel, GR-HMC had 0.0% divergent simulations at step sizes 0.05-0.09 where RHMC diverged in 2.0%-35.4%, and 1.8% at 0.10 versus 44.6% for RHMC.
- On hierarchical Bayesian logistic regression (26D) with 1,000 samples, GR-HMC ESS at step size 0.08 was 210.2 versus 179.2 for RHMC and 14.1 for SR-HMC.
- On log-GCPP (64D), GR-HMC ESS at step size 0.2 was 4446.3 versus 1490.6 for RHMC; RHMC was best only at 0.1 on HBLR and SR-HMC at 0.1 and 0.5 on logGCPP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use position-dependent velocity bounds in place of unbounded Riemannian momentum | RHMC · 2D Neal's funnel, step size 0.05–0.10 | divergent simulations: 2.0% -> 0.0%; 6.4% -> 0.0%; 13.2% -> 0.0%; 21.8% -> 0.0%; 35.4% -> 0.0%; 44.6% -> 1.8% | Helped |
| Use GR-HMC instead of RHMC | HBLR · HBLR | ESS: RHMC: 19.3, 52.3, 105.8, 179.2, 258.7; GR-HMC: 20.6, 80.7, 147.5, 210.2, 217.8 | Mixed |
| Use GR-HMC instead of RHMC | HMC with diagonal SoftAbs approximation · logGCPP | ESS: RHMC: 272.0, 1490.6, 4756.8, 4.7, 3.0; GR-HMC: 160.9, 4446.3, 4764.1, 4456.7, 127.1 | Mixed |
| Use general relativistic momentum rather than special relativistic momentum | HMC with multinomial trajectory sampling · Neal's funnel | ESS | Helped |
Surprising findings
- On HBLR at step size 0.1, RHMC outperforms GR-HMC, which the authors attribute to RHMC's acceptance rate not dropping quickly with step size on that posterior.
Limitations
- Full Hessian-based GR-HMC remains computationally expensive (cubic cost), so real-world experiments use diagonal approximations.
- Performance on the HBLR problem at large step size is not uniformly better than RHMC.
- Hyperparameters m and c must be chosen, and the sweep over them was not done for real-world datasets due to computation limits.
Experimental setup
Datasets: Neal's funnel (10D and 2D) hierarchical Bayesian logistic regression (26D) log-Gaussian Cox point process (64D)
Benchmarks: Neal's funnel hierarchical Bayesian logistic regression log-Gaussian Cox point process
Metrics: percentage of divergent simulations total travel distance final and average energy difference effective sample size MH acceptance ratio
Baselines: HMC SR-HMC RHMC dimension-wise SR-HMC
Builds on: Lu et al. (2017) relativistic Monte Carlo Girolami & Calderhead (2011) Riemannian HMC Betancourt (2013) SoftAbs metric HMC with generalized leapfrog
Closest papers by meaning
- Evaluating the Implicit Midpoint Integrator for Riemannian Hamiltonian Monte Carlo (ICML 2021 · similarity 0.89)
- Unbiased constrained sampling with Self-Concordant Barrier Hamiltonian Monte Carlo (NeurIPS 2023 · similarity 0.89)
- Robust Monte Carlo Sampling using Riemannian Nosé-Poincaré Hamiltonian Dynamics (ICML 2016 · similarity 0.88)
- Magnetic Hamiltonian Monte Carlo (ICML 2017 · similarity 0.87)
- Metropolis Adjusted Microcanonical Hamiltonian Monte Carlo (NeurIPS 2025 · similarity 0.84)
- Riemannian Proximal Sampler for High-accuracy Sampling on Manifolds (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.