Research map / Probabilistic and scientific modeling / Variational and generative sampling
Parallel Affine Transformation Tuning of Markov Chain Monte Carlo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes parallel affine transformation tuning (PATT), which runs multiple parallel chains of a base MCMC sampler (mainly Gibbsian polar slice sampling) in a latent space whose affine transformation is periodically re-learned from pooled samples, and shows strong efficiency on several Bayesian inference targets.
Problem
Generic MCMC samplers perform poorly on targets with unknown covariance structure, off-center mass, or varied tails, especially in high dimensions, leading to slow convergence and long autocorrelations.
Approach
Adaptively learn a bijective affine transformation (centering and covariance/variance adjustment) from samples pooled across parallel chains, sample in the transformed latent space with a base sampler (GPSS or ESS), and update the transformation only on a sparse update schedule to limit synchronization idling.
Key results
- On the multivariate exponential model (d=50), PATT-GPSS achieves 6.68 TDE/ES versus 151.13 for AdaRWM and 16657.77 for HRUSS.
- On Bayesian logistic regression, PATT-ESS achieves the lowest TDE/ES on Credit (1.8) and Breast (43.4), outperforming all competitors.
- On GP hyperparameter inference (d=30), NUTS beats PATT in TDE/ES, but PATT-ESS and PATT-GPSS are roughly even with NUTS in ES/s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PATT-GPSS instead of AdaRWM | Multivariate exponential Bayesian inference, d=50 · Multivariate exponential Bayesian inference, d=50 | TDE/ES: 151.13 -> 6.68 | Helped |
| Use PATT-ESS instead of AdaRWM | Multivariate exponential Bayesian inference, d=50 · Multivariate exponential Bayesian inference, d=50 | TDE/ES: 151.13 -> 20.10 | Helped |
| Use PATT-ESS instead of Stan's NUTS | BLR Credit · BLR Credit | TDE/ES: 50.7 -> 1.8 | Helped |
| Use PATT-ESS instead of AdaRWM | BLR Breast · BLR Breast | TDE/ES: 151.0 -> 43.4 | Helped |
| Use PATT-ESS instead of Stan's NUTS | BLR Pima · BLR Pima | TDE/ES: 29.8 -> 5.9 | Helped |
| Use PATT-ESS instead of Stan's NUTS | BLR Wine · BLR Wine | TDE/ES: 227.5 -> 12.6 | Helped |
| Use PATT-ESS instead of PATT-GPSS | BLR Credit, Breast, Pima, and Wine · BLR datasets | TDE/ES: 7.6 -> 1.8; 71.7 -> 43.4; 19.1 -> 5.9; 28.3 -> 12.6 | Helped |
| Use Stan's NUTS instead of PATT-ESS | Gaussian process regression hyperparameter inference, US census data · Gaussian process regression hyperparameter inference, US census data | TDE/ES: 21.95 -> 15.30 | Helped |
| Use PATT-ESS instead of PATT-GPSS | Gaussian process regression hyperparameter inference, US census data · Gaussian process regression hyperparameter inference, US census data | TDE/ES: 50.61 -> 21.95 | Helped |
Surprising findings
- PATT-ESS outperforms PATT-GPSS in TDE/ES on logistic regression despite GPSS being the default recommendation.
- The authors note that PATT may need many iterations in very high dimensions before approaching asymptotic performance, with slow learning of the transformation.
Limitations
- Learning speed of the transformation deteriorates with dimension, so very high-dimensional targets may need many iterations.
- Covariance adjustments are computationally costly (O(d^3) for Cholesky and inverse) and need p or update spacing scaled with d.
- Variance and covariance adjustments can be ill-defined when samples are degenerate, requiring care with initialization or regularization.
Experimental setup
Datasets: synthetic data Credit Breast cancer Pima Wine US census data
Benchmarks: Bayesian multivariate exponential model (d=50) Bayesian logistic regression on Credit, Breast, Pima, Wine Gaussian process hyperparameter inference on US census data (d=30)
Metrics: target density evaluations per effective sample (TDE/ES) effective samples per second (ES/s) mean integrated autocorrelation time mean step size TDE per iteration samples per second
Baselines: HRUSS AdaRWM GESS NUTS PATT-ESS
Builds on: Gibbsian polar slice sampling elliptical slice sampling adaptive random walk Metropolis No-U-Turn Sampler AirMCMC
Closest papers by meaning
- Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization (ICML 2025 · similarity 0.87)
- Gradient-based Adaptive Markov Chain Monte Carlo (NeurIPS 2019 · similarity 0.87)
- AutoStep: Locally adaptive involutive MCMC (ICML 2025 · similarity 0.87)
- Parallelizing MCMC Across the Sequence Length (NeurIPS 2025 · similarity 0.86)
- Ai-sampler: Adversarial Learning of Markov kernels with involutive maps (ICML 2024 · similarity 0.85)
- Reverse Diffusion Sequential Monte Carlo Samplers (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.