Research map / Probabilistic and scientific modeling / Variational and generative sampling

To discretize continually: Mean shift interacting particle systems for Bayesian inference

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Hamiltonian Monte Carlo · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes mean shift interacting particle systems (MSIP), a regularized, normalization-invariant MMD-minimizing particle method that yields weighted quadrature rules for unnormalized densities, with gradient-free and gradient-informed variants.

Problem

Constructing weighted quadrature rules (small weighted particle sets) that approximate expectations under a target known only up to its normalizing constant, where MMD minimization is considered infeasible.

Approach

Extends the mean shift interacting particle (MSIP) algorithm from empirical targets to unnormalized densities by estimating the kernel embeddings v0 and v1 via inner Gaussian quadrature (Fredholm, Stein, or gradient-free estimators) and adds regularization; the resulting fixed-point dynamics are invariant to the target's normalizing constant, and the regularized objective is a penalized MMD.

Key results

• On the five-component anisotropic Gaussian mixture, MSIP-Fredholm captures all modes while SVGD and ALDI suffer mode collapse, with MSIP outperforming by a significant MMD margin especially at large particle counts.
• MSIP variants achieve the lowest or near-lowest KSD on most synthetic targets; e.g., on Funnel50 MSIPGI-1 gets 46.68 vs SVGD 242.5 and ALDI failing.
• On TwoMoons201 (Bayesian neural network, d=201), MSIPF log-likelihood error is 258.60 vs SVGD 13653, showing faster convergence than SVGD.

Surprising findings

• SVGD frequently underperforms MSIP on several Gaussian mixtures, and small KSD values do not guarantee global mode coverage on multi-modal targets.
• The gradient-free MSIP-GF is remarkably competitive across most synthetic experiments despite using no gradients.

Limitations

• No theoretical convergence guarantees are established; the nonconvex optimization remains analytically challenging.
• Performance depends on hyperparameters such as kernel bandwidth and step size, which are set per example.
• KSD-based evaluation can fail to reflect mode allocation on multi-modal targets.

Experimental setup

Datasets: GMM2 GMM5 GMM10 GMM20 Joker Funnel Himmelblau Schools (posteriordb) COVID (posteriordb) CovType ABPDE TwoMoons

Benchmarks: GMM2 GMM5 GMM10 GMM20 Joker Funnel2 Funnel5 Funnel10 Funnel20 Funnel50 Himmelblau Schools10

Metrics: kernel Stein discrepancy log-likelihood MMD

Baselines: SVGD ALDI CBS Polarized CBS multi-start gradient ascent

Builds on: mean shift interacting particles Stein variational gradient descent ALDI consensus-based sampling

Closest papers by meaning

• Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.87)
• Particle Semi-Implicit Variational Inference (NeurIPS 2024 · similarity 0.87)
• Sampling in Unit Time with Kernel Fisher-Rao Flow (ICML 2024 · similarity 0.86)
• Path-Guided Particle-based Sampling (ICML 2024 · similarity 0.86)
• A Mean-Field Framework for Inference-Time Distributional Control of Diffusion Models (NeurIPS 2026 · similarity 0.86)
• Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Models (ICML 2025 · similarity 0.86)