Research map / Probabilistic and scientific modeling / Variational and generative sampling
The Polynomial Stein Discrepancy for Assessing Moment Convergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the polynomial Stein discrepancy (PSD), a linear-time Langevin-Stein discrepancy built from r-th order monomial test functions, with a bootstrap goodness-of-fit test that detects moment discrepancies for Gaussian targets.
Problem
Assessing sample quality of biased MCMC samplers (e.g. SG-MCMC) against a posterior is costly with KSD (quadratic in n), linear-time KSD alternatives lose power, suffer from tuning needs, curse of dimensionality, and fail to detect moment convergence.
Approach
Replace the RKHS Stein set with the span of polynomials of degree at most r under the Langevin Stein operator; the resulting discrepancy is the norm of the vector of sample-averaged Stein-transformed monomials, giving a closed-form, linear-time statistic that is zero iff the first r moments match for Gaussian targets.
Key results
- PSD with r=4 is the only method consistently achieving power near 1 in the Gaussian goodness-of-fit experiments with variance and kurtosis discrepancies, while keeping type I error near 0.05 in most settings.
- In the RBM goodness-of-fit test, PSD r=2 gives null rejection rate 1.00 at perturbation 0.02, versus 0.99 for IMQ KSD and 0.48 for RFSD.
- For SGLD step-size selection, PSD (r=2,3,4) selects step size 0.005 like IMQ KSD, and PSD is reported around 70 times faster than KSD and around 7 times faster than RFSD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PSD with r=2 instead of linear-time competitors for the RBM perturbation test | RBM goodness-of-fit testing · RBM, perturbation 0.02 | null rejection rate: 0.70 -> 1.00 | Helped |
| Increase polynomial order from r=1 to r=2 (ablation) | PSD · RBM, perturbation 0.02 | null rejection rate: 0.51 -> 1.00 | Helped |
| Increase polynomial order from r=1 to r=4 to capture fourth-moment discrepancies (ablation) | PSD · Unit Gaussian versus variance-perturbed Gaussian, Student-t, and Laplace tests | statistical power | Helped |
| Use PSD with r=1 when the discrepancy is in second- or fourth-order moments (ablation) | PSD · Gaussian-target goodness-of-fit tests | statistical power | Hurt |
| Use PSD with r=2 or r=3 for fourth-moment discrepancies (ablation) | PSD · Student-t and Laplace goodness-of-fit tests | statistical power | Hurt |
| Replace the PSD bootstrap test with the asymptotic test using samples from Q (ablation) | PSD goodness-of-fit testing · Unit Gaussian and misspecified-distribution tests | statistical power | Hurt |
| Remove interaction terms from PSD (ablation) | PSD · SGLD and logistic regression sample-quality examples | performance | No clear effect |
| Use PSD with r=2, r=3, or r=4 to select the SGLD step size (ablation) | SGLD hyperparameter selection · Bimodal Gaussian mixture posterior | step size selected | Helped |
| Use PSD rather than KSD to compute the sample-quality discrepancy | SGLD sample-quality assessment · Bimodal Gaussian mixture posterior | runtime speedup: around 70 times faster | Helped |
| Use PSD with r≥2 instead of KSD or PSD with r=1 to assess ULA moment estimation (ablation) | ULA sample-quality assessment · Logistic regression with sparsity-inducing prior | identification of best step size | Helped |
| Use higher polynomial order rather than r=1 (ablation) | PSD for ULA · Logistic regression big-data example | identification of best step size | Helped |
Surprising findings
- KSD outperforms PSD with r=1,2,3 for Student-t and Laplace examples because the discrepancy lies in kurtosis (fourth-order moment).
- PSD without interaction terms performs remarkably similarly to full PSD on SGLD and logistic regression examples.
Limitations
- PSD is not fully convergence-determining because it uses a finite-dimensional, non-characteristic function class; guarantees of moment detection hold mainly for Gaussian targets.
- Likely less effective for heavy-tailed targets lacking finite moments (e.g. Cauchy), and for discrepancies in moments above order r.
- Asymptotic test has poor type I error control and computing eigenvalues costs O(J^3); bootstrap test is recommended instead.
Experimental setup
Datasets: synthetic Gaussian samples restricted Boltzmann machine samples bimodal Gaussian mixture posterior samples logistic regression posterior samples
Benchmarks: Gaussian goodness-of-fit (variance-perturbed, Student-t, Laplace) RBM goodness-of-fit SGLD step size selection on Gaussian mixture runtime ULA logistic regression step selection
Metrics: type I error rate statistical power null rejection rate runtime
Baselines: IMQ KSD Gauss KSD FSSD-opt RFSD
Builds on: kernel Stein discrepancy Langevin Stein operator finite set Stein discrepancy random Fourier feature Stein discrepancy zero variance control variates
Closest papers by meaning
- Measuring Sample Quality with Kernels (ICML 2017 · similarity 0.92)
- Random Feature Stein Discrepancies (NeurIPS 2018 · similarity 0.92)
- Stochastic Stein Discrepancies (NeurIPS 2020 · similarity 0.92)
- Gradient-Free Kernel Stein Discrepancy (NeurIPS 2023 · similarity 0.91)
- Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy (ICML 2023 · similarity 0.89)
- A Kernelized Stein Discrepancy for Goodness-of-fit Tests (ICML 2016 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.