Research map / Probabilistic and scientific modeling / Variational and generative sampling
Variance Reduction and Quasi-Newton for Particle-Based Variational Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| variance reduction (SVRG, SPIDER, SQN-VR) applied to SVGD-based ParVI | SVGD with linear kernel, Bayesian linear regression · noise dataset | MMD (log10): about 10^-0.85 best baseline vs 10^-1.38 to 10^-1.63 | Helped |
| variance reduction alone (SVRG) without quasi-Newton | SVGD with linear kernel, Bayesian linear regression · parkinson dataset | MMD (log10): SVRG ~10^-1.06 vs SGD/AdaGrad ~10^-1.3 | No clear effect |
| adding quasi-Newton (L-BFGS) preconditioning to variance reduction (SQN-VR) (ablation) | SVGD with linear kernel, Bayesian linear regression · parkinson dataset | MMD (log10): SQN-VR 10^-1.56 vs others ~10^-1.3 | Helped |
| adding quasi-Newton (L-BFGS) preconditioning to variance reduction (SQN-VR) (ablation) | SVGD with linear kernel, Bayesian linear regression · toms dataset (ill-conditioned, cond 45,923) | MMD (log10): SQN-VR 10^-1.34 vs AdaGrad 10^-0.52 and others no better than 10^-0.18 | Helped |
| variance reduction (SVRG, SPIDER, SQN-VR) applied to SVGD | SVGD with linear kernel, Bayesian logistic regression · MNIST 7 vs 9 | MMD (log10): about 10^-1.85 vs AdaGrad 10^-0.81 and SGD 10^-1.01 | Helped |
| variance reduction / quasi-Newton applied to SVGD | SVGD with linear kernel, Bayesian logistic regression · covtype | MMD (log10): SQN-VR and SPIDER ~10^-1.7 vs SGD 10^-1.19 and AdaGrad 10^-0.19 | Helped |
Closest papers by meaning
- Particle Semi-Implicit Variational Inference (NeurIPS 2024 · similarity 0.90)
- Functional Gradient Flows for Constrained Sampling (NeurIPS 2024 · similarity 0.90)
- Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.89)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.89)
- ELBOing Stein: Variational Bayes with Stein Mixture Inference (ICLR 2025 · similarity 0.88)
- GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.