Research map / Probabilistic and scientific modeling / Variational and generative sampling
Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics
Kamélia Daudel, Joe Benton, Yuyang Shi, Arnaud Doucet
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase dimension d with fixed N (log-normal weights, B_d = sqrt(d)) | VR-IWAE bound · Gaussian example, d in {10,100,1000} | variational gap | Hurt |
| Train phi so that B_d/sqrt(d) decreases from 1 to 0 | VR-IWAE bound · Gaussian example, alpha=0.2, d=1000 | variational gap approximation by fixed-d 1/N asymptotics | Improved |
| Increase alpha in [0,1) for the bound | VR-IWAE bound · Gaussian example | weight-collapse limitation in high dimension | No clear effect |
| Use alpha > 0 instead of alpha = 0 (IWAE) | reparameterized inference network gradient · Linear Gaussian, d in {20,100}, sigma_pert=0.01 | SNR of inference network gradient | Improved |
| Increase alpha | VR-IWAE gradient estimators · Linear Gaussian, low-medium d | SNR of generative and inference networks | Improved |
| Move parameters away from optimum (sigma_pert=0.5) or increase d to 1000 | SNR gain from alpha > 0 · Linear Gaussian | SNR improvement | Hurt |
| Use doubly-reparameterized gradient estimator for phi | VR-IWAE inference network gradients · Linear Gaussian | SNR decay | Improved |
| Increase alpha | log-likelihood estimation · Linear Gaussian | MSE of log-likelihood estimate | No clear effect |
| Increase alpha | theta gradient estimates · Linear Gaussian | MSE of theta gradient | Improved |
| Use optimal parameters (sigma_pert=0) with increasing d | VR-IWAE bound · Linear Gaussian | VR-IWAE bound | Hurt |
| Let N grow faster than d^(1/3) | asymptotic regime of the bound · Linear Gaussian | validity of the high-dimensional asymptotic | Hurt |
Closest papers by meaning
- Hierarchical Importance Weighted Autoencoders (ICML 2019 · similarity 0.89)
- Monotonic Alpha-divergence Minimisation for Variational Inference (JMLR 2023 · similarity 0.88)
- Inference Suboptimality in Variational Autoencoders (ICML 2018 · similarity 0.88)
- Recursive Inference for Variational Autoencoders (NeurIPS 2020 · similarity 0.87)
- Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families (NeurIPS 2024 · similarity 0.86)
- Variational Learning of Fractional Posteriors (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
