Research map / Optimization / Optimizers
A Gradient Based Strategy for Hamiltonian Monte Carlo Hyperparameter Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SKSD-based automatic scale tuning of the initial distribution to maxELT (ablation) | maxELT with alpha=0 initial distribution · MNIST and Fashion MNIST VAE | test marginal log-likelihood: MNIST -83.48 to -81.91; Fashion -104.08 to -103.18 | Helped |
| Add SKSD scale tuning to maxELT with alpha=1 initial distribution (ablation) | maxELT alpha=1 on 2D synthetic distributions · 2D synthetic distributions | KSD (lower is better): Dual Moon 0.8637 -> 0.2528; Gaussian 0.0009 -> 0.0009; Mixture 0.0010 -> 0.0019 | Mixed |
| Add SKSD scale tuning to maxELT with alpha=1 initial distribution (ablation) | sparse signal recovery · Bayesian compressed sensing | test log likelihood: slight degradation | Hurt |
| Use alpha=1 instead of alpha=0 for initial distribution training without scale tuning (ablation) | maxELT on VAE · MNIST and Fashion MNIST | test marginal log-likelihood: MNIST -83.48 to -82.46; Fashion -104.08 to -103.57 | Helped |
| Use alpha=0 mass-seeking narrow initial distribution without scale tuning | maxELT on Alanine Dipeptide · Alanine Dipeptide Ramachandran KL | KL divergence: worse than pa=0.65 baseline | Hurt |
| Use maxELT with SKSD (ours) instead of tuning HMC via grid search or acceptance 0.65 for VAE-based HMC | HMC hyperparameters on VAE · MNIST and Fashion MNIST | test marginal log-likelihood: MNIST -81.91 vs Ruiz & Titsias -82.45; Fashion -102.29 vs Ruiz & Titsias -105.13 | Helped |
| Tune HMC step sizes with maxELT (expected log target) rather than heuristic tuning on 2D targets | HMC on 2D synthetic distributions · 2D synthetic distributions | KSD (lower is better): Dual Moon: maxELT α=0 SKSD 0.1684 vs NUTS 0.2326 and Ruiz & Titsias 1.6290 | Helped |
| Use maxELT with SKSD vs Ruiz & Titsias on simple targets | 2D synthetic distributions · Gaussian and Laplace targets | KSD (lower is better): Ruiz & Titsias 0.0003 vs ours 0.0008 (Gaussian) | Hurt |
| Replace grid search/acceptance-rate tuning with maxELT for Alanine Dipeptide proposal from ML training | HMC on Alanine Dipeptide · Alanine Dipeptide marginal distributions | Wilcoxon p-value vs baseline: ML: grid search p=0.010, pa=0.65 p=5.1e-5 | Helped |
Closest papers by meaning
- On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients (ICML 2021 · similarity 0.92)
- Entropy-based adaptive Hamiltonian Monte Carlo (NeurIPS 2021 · similarity 0.91)
- Generalizing Hamiltonian Monte Carlo with Neural Networks (ICLR 2018 · similarity 0.90)
- Stochastic Gradient Hamiltonian Monte Carlo Methods with Recursive Variance Reduction (NeurIPS 2019 · similarity 0.90)
- Metropolis Adjusted Microcanonical Hamiltonian Monte Carlo (NeurIPS 2025 · similarity 0.89)
- Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.