Research map / Probabilistic and scientific modeling / Variational and generative sampling
Sampling in Unit Time with Kernel Fisher-Rao Flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Kernel Fisher-Rao Flow, a gradient-free mean-field ODE and interacting particle system (plus a stochastic variant) that transports samples from a reference to a target along the geometric mixture path in unit time, using an RKHS ansatz for the velocity field.
Problem
Sampling from an unnormalized target density when only the target-to-reference density ratio (likelihood) can be evaluated and gradients are unavailable; existing dynamic samplers either need gradients or run for infinite time.
Approach
Solve a weak-form Poisson equation for a velocity field along the tempered-likelihood (Fisher-Rao gradient flow) path, restricted to an RKHS, yielding a closed-form gradient-free particle ODE; the same system is recovered in discrete time as a linearized sample-driven optimal transport (Monge-Ampere) update.
Key results
- KFRFlow and KFRFlow-I generally achieve lower KSD than gradient-free EKI, EKS and CBS across ensemble sizes J in {100,400} and step counts N in 2^1..2^8 on the two-dimensional posterior examples.
- KFRFlow-I is stable across all (J, N) settings on the donut example, including some cases where gradient-based KFRD and SVGD are unstable.
- On funnel distributions in d=5 to 20, KFRFlow-I produces better KSD than gradient-free CBS and its quality does not degrade noticeably with dimension.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use KFRFlow or KFRFlow-I instead of EKI, EKS, or CBS | Gradient-free sampling for two-dimensional Bayesian posteriors · Donut, Butterfly, and Spaceships | KSD | Helped |
| Use KFRFlow-I instead of KFRFlow | Sampling two-dimensional Bayesian posteriors · Donut, Butterfly, and Spaceships | stability | Helped |
| Use KFRD instead of ULA | Gradient-based sampling for two-dimensional Bayesian posteriors · Donut, Butterfly, and Spaceships | KSD | No clear effect |
| Use KFRD instead of SVGD | Gradient-based sampling for two-dimensional Bayesian posteriors · Donut, Butterfly, and Spaceships | KSD | Hurt |
| Use KFRFlow or KFRFlow-I instead of SVGD | Sampling with small step counts · Two-dimensional Bayesian posteriors | KSD | Mixed |
| Use KFRFlow-I instead of CBS | Sampling funnel distributions · Funnels (d = 5, 10, 15, 20) | KSD | Helped |
| Increase the funnel dimension | KFRFlow-I · Funnel distributions | KSD | No clear effect |
| Use fourth-order Adams-Bashforth instead of forward Euler (ablation) | KFRFlow · Two-dimensional Bayesian posteriors | KSD | Helped |
| Use Adams-Bashforth 4 instead of forward Euler (ablation) | SVGD · Two-dimensional Bayesian posteriors | KSD | Hurt |
| Add stochastic noise and score-based drift (ablation) | KFRFlow (forming KFRD) · Sampling from the geometric mixture | sample quality | Helped |
| Use KFRFlow-I instead of an Euler discretization of KFRFlow | Sampling at large step sizes · Two-dimensional Bayesian posteriors | stability | Helped |
| Use KFRFlow-I instead of serial random-walk Metropolis | Sampling two-dimensional Bayesian posteriors · Most settings with N > 8 | KSD | Helped |
Surprising findings
- Parallel random walk Metropolis outperforms KFRFlow variants for sufficiently large step counts N on the two-dimensional posteriors.
- For funnels, KFRFlow-I's samples are better than serial random walk Metropolis for d>5, becoming comparable to parallel RWM by d=20.
Limitations
- Vanilla implementation has O(J^3) cost per step; scalability to high dimensions and large ensembles is not demonstrated.
- The RKHS ansatz means the particle distribution rho_t only approximates the target path pi_t; approximation error and sample complexity are not analyzed.
- Numerical instability requires diagonal regularization of M_t and step-size tuning, with hyperparameters selected via direct search against the target.
Experimental setup
Datasets: two-dimensional Bayesian posterior examples (donut, butterfly, spaceships) d-dimensional funnel distributions
Benchmarks: two-dimensional Bayesian posteriors (donut, butterfly, spaceships) funnel distributions d=5,10,15,20
Metrics: kernel Stein discrepancy (IMQ kernel, h=1) runtime
Baselines: EKI EKS CBS SVGD ULA random walk Metropolis KFRD
Builds on: Stein variational gradient descent ensemble Kalman inversion sample-driven optimal transport annealed importance sampling
Closest papers by meaning
- NETS: A Non-equilibrium Transport Sampler (ICML 2025 · similarity 0.88)
- Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling (NeurIPS 2026 · similarity 0.88)
- Neural Sampling from Boltzmann Densities: Fisher-Rao Curves in the Wasserstein Geometry (ICLR 2025 · similarity 0.87)
- Path-Guided Particle-based Sampling (ICML 2024 · similarity 0.87)
- Parametric model reduction of mean-field and stochastic systems via higher-order action matching (NeurIPS 2024 · similarity 0.87)
- To discretize continually: Mean shift interacting particle systems for Bayesian inference (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.