Research map / Probabilistic and scientific modeling / Variational and generative sampling

Sobolev Regularized MMD Gradient Flow

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Posterior sampling with diffusion guidance · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Sobolev-regularized MMD (SrMMD) gradient flow, penalizing the witness function's gradient norm, with global convergence guarantees in continuous and discrete time, applicable to both generative modelling and sampling via Stein kernels.

Problem

MMD and KSD gradient flows are non-convex (not geodesically convex) in Wasserstein space, lacking global convergence guarantees; existing sampling rates depend on isoperimetric constants that blow up for multimodal targets.

Approach

Replace the MMD witness with one penalized by a gradient norm (Sobolev) term, giving a closed-form witness (S_mu + lambda Id)^{-1}(m_mu - m_pi) and a flow that converges exponentially in MMD/KSD without isoperimetric assumptions, under a source condition on the trajectory.

Key results

• On Gaussian mixture and Swiss roll generative modelling, SrMMD substantially outperforms vanilla MMD flow in MMD and W2 across 10 seeds, performing comparably to DrMMD.
• On sampling a 10-component Gaussian mixture, SrMMD attains the fastest KSD decay and lowest final KSD among KSD flow, SVGD and R-SVGD.
• Theory: continuous-time MMD^2 decays as exp(-2T) up to residual O(lambda^alpha); discrete-time SrMMD descent achieves O(delta^{-1/alpha}) iteration complexity versus O(delta^{-1}) for SVGD.

Surprising findings

• SrMMD flow significantly improves on KSD flow in Bayesian logistic regression while matching SVGD, despite sampling-side convergence being stated only in KSD.
• Original KSD flow's Stein-Poincare convergence condition is claimed to never hold.

Limitations

• Each iteration requires inverting an Nd x Nd matrix, costing O(N^3 d^3).
• Source condition m_mu - m_pi in Ran(S_mu^{alpha/2}) is hard to verify in practice; iteration complexity saturates at alpha <= 1.
• Particle implementation's convergence to the population scheme is not analyzed; experiments limited to small-scale benchmarks.

Experimental setup

Datasets: Gaussian mixture (synthetic) Swiss roll (synthetic) mean-field student-teacher spherical data Bayesian logistic regression datasets (Breast Cancer, Ionosphere, German Credit, Covtype)

Benchmarks: Gaussian mixture Swiss roll student-teacher network color transfer Bayesian logistic regression

Metrics: MMD W2 KSD test accuracy test log-likelihood

Baselines: MMD flow DrMMD flow HrMMD flow KSD flow SVGD R-SVGD

Builds on: MMD flow KSD flow DrMMD flow Sobolev descent SVGD

Closest papers by meaning

• Generative Sliced MMD Flows with Riesz Kernels (ICLR 2024 · similarity 0.90)
• Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel (ICLR 2024 · similarity 0.89)
• Deep MMD Gradient Flow without adversarial training (ICLR 2025 · similarity 0.89)
• Functional Gradient Flows for Constrained Sampling (NeurIPS 2024 · similarity 0.87)
• Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization (NeurIPS 2025 · similarity 0.87)
• Flow matching achieves almost minimax optimal convergence (ICLR 2025 · similarity 0.86)