Research map / Probabilistic and scientific modeling / Variational and generative sampling
On the difficulty of unbiased alpha divergence minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use α ≠ 0 instead of the α → 0 limit | Unbiased alpha-divergence gradient estimators · Fully-factorized distributions and Gaussian distributions | gradient SNR | Hurt |
| Use α → 0 rather than α ≠ 0 | Unbiased alpha-divergence gradient estimators · Fully-factorized distributions | gradient SNR | Helped |
| Use α → 0 rather than α ≠ 0 | Unbiased alpha-divergence gradient estimators · Full-rank Gaussians | gradient SNR | Helped |
| Use double reparameterization instead of reparameterization (ablation) | Unbiased alpha-divergence optimization · Gaussian target and factorized Gaussian approximation | final optimization performance | Helped |
| Increase α from 0 toward 0.3 | Unbiased alpha-divergence optimization · Australian logistic regression, d=14 | optimization convergence | Hurt |
| Increase α from 0 toward the tested nonzero values | Unbiased alpha-divergence optimization · Iris logistic regression, d=4 | optimization convergence | No clear effect |
| Increase dimensionality while α ≠ 0 | Unbiased alpha-divergence optimization with double reparameterization · Standard Gaussian target and factorized Gaussian approximation | optimization convergence | Hurt |
| Use Adam instead of SGD (ablation) | Unbiased alpha-divergence optimization · Gaussian and logistic-regression experiments | optimization convergence | No clear effect |
| Increase the number of gradient samples | Unbiased alpha-divergence optimization · Standard Gaussian target, d=32, α=1.5 | optimization convergence | No clear effect |
Closest papers by meaning
- Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families (NeurIPS 2024 · similarity 0.89)
- Near-Optimality of Contrastive Divergence Algorithms (NeurIPS 2024 · similarity 0.88)
- Variance-based Regularization with Convex Objectives (NeurIPS 2017 · similarity 0.87)
- Demystifying SGD with Doubly Stochastic Gradients (ICML 2024 · similarity 0.86)
- Least squares variational inference (NeurIPS 2025 · similarity 0.86)
- Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.