Research map / Probabilistic and scientific modeling / Variational and generative sampling
GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GAD-PVI, a particle-based variational inference framework combining accelerated (momentum) position updates and dynamic weight adjustment by discretizing a semi-Hamiltonian flow on an Information-Fisher-Rao space, yielding nine instances across three information metrics.
Problem
Existing ParVI methods either accelerate particle position updates or adjust particle weights dynamically, but not both; the goal is to get both benefits for faster convergence and lower approximation error in Bayesian inference.
Approach
Augments the information-space gradient flow with an orthogonal Fisher-Rao structure (IFR space) and simulates a semi-Hamiltonian flow in it, which gives particle ODEs for position, velocity and weight; explicit Euler discretization yields GAD-PVI, which is compatible with KL-BLOB, KL-GFSD and KSD-KSDD smoothing.
Key results
- On the LIDAR GP task with W2 metric, WGAD-CA-BLOB reached 1.274e-01 vs 1.285e-01 for DPVI-CA-BLOB and 1.570e-01 for classical ParVI-BLOB (lower is better).
- On BNN regression, WGAD-CA-BLOB achieved the lowest test RMSE on Concrete (6.047 vs 6.323 for SVGD) and kin8nm (7.629e-02 vs 8.020e-02 for SVGD).
- The paper reports nine GAD-PVI instances and claims faster convergence and reduced approximation error than state-of-the-art ParVI methods across synthetic and real-world tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WGAD-CA instead of DPVI-CA with BLOB smoothing | GP LIDAR · GP LIDAR | W2 distance: 1.285e-01 $\pm $ 2.960e-04 -> 1.274e-01 $\pm $ 2.964e-04 | Helped |
| Use WGAD-CA instead of DPVI-CA with GFSD smoothing | GP LIDAR · GP LIDAR | W2 distance: 1.638e-01 $\pm $ 4.332e-04 -> 1.626e-01 $\pm $ 4.842e-04 | Helped |
| Combine accelerated position updates and dynamic weight adjustment | GAD-PVI · BNN Concrete | test RMSE: 6.063e+00 -> 6.047e+00 | Helped |
| Combine accelerated position updates and dynamic weight adjustment | GAD-PVI · BNN kin8nm | test RMSE: 7.690e-02 -> 7.629e-02 | Helped |
| Combine accelerated position updates and dynamic weight adjustment | GAD-PVI · BNN RedWine | test RMSE: 6.264e-01 -> 6.260e-01 | Helped |
| Combine accelerated position updates and dynamic weight adjustment | GAD-PVI · BNN space | test RMSE: 8.775e-02 -> 8.716e-02 | Helped |
| Use continuous weight adjustment instead of Duplicate/Kill (ablation) | WGAD · GP LIDAR, BLOB smoothing | W2 distance: 1.561e-01 $\pm $ 1.155e-03 -> 1.274e-01 $\pm $ 2.964e-04 | Helped |
| Use continuous weight adjustment instead of Duplicate/Kill (ablation) | WGAD · GP LIDAR, GFSD smoothing | W2 distance: 2.142e-01 $\pm $ 1.501e-03 -> 1.626e-01 $\pm $ 4.842e-04 | Helped |
| Use continuous weight adjustment instead of Duplicate/Kill (ablation) | GAD-PVI · Gaussian mixture model | W2 distance | Helped |
Limitations
- Full Hamiltonian IFR flow is intractable to discretize, so the method uses an approximating semi-Hamiltonian flow that drops the Fisher-Rao kinetic energy term
- KSDD-smoothed instances were excluded from experiments due to computational cost and instability
- Hyperparameters were tuned by grid search per algorithm
Experimental setup
Datasets: LIDAR Concrete kin8nm RedWine space
Benchmarks: LIDAR GP regression Concrete BNN kin8nm BNN RedWine BNN space BNN 10-D Gaussian mixture
Metrics: W2 distance RMSE
Baselines: SVGD ParVI-BLOB ParVI-GFSD WNES-BLOB WNES-GFSD WAIG-BLOB WAIG-GFSD DPVI-CA DPVI-DK
Builds on: Stein Variational Gradient Descent BLOB GFSD DPVI WAIG WNES ACCEL
Closest papers by meaning
- Particle Semi-Implicit Variational Inference (NeurIPS 2024 · similarity 0.91)
- Understanding and Accelerating Particle-Based Variational Inference (ICML 2019 · similarity 0.90)
- Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.86)
- Functional Gradient Flows for Constrained Sampling (NeurIPS 2024 · similarity 0.86)
- Online Variational Sequential Monte Carlo (ICML 2024 · similarity 0.85)
- Kernel Semi-Implicit Variational Inference (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.