Research map / Probabilistic and scientific modeling / Variational and generative sampling
Minimizing $f$-Divergences by Interpolating Velocity Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes estimating Wasserstein gradient flow velocity fields for f-divergences directly via Nadaraya-Watson and local linear interpolation (using a mirror-divergence variational objective), with consistency guarantees, applied to domain adaptation and missing data imputation.
Problem
Minimizing f-divergences between particle and target distributions with WGF requires the velocity field grad(h∘r), but density ratio estimation followed by differentiation overfits and gives inaccurate gradients, especially with only samples from the target.
Approach
Directly estimate the WGF velocity field by local (non-parametric) interpolation: NW interpolation of the backward KL field when grad log p is known, and a local linear fit obtained by maximizing a variational lower bound of the mirror divergence, whose slope approximates grad(h∘r) using only samples from p and q.
Key results
- On Office-Caltech-10 domain adaptation, WGF achieves best accuracy in most of 12 adaptation pairs versus JDOT and MMDFlow (e.g., dslr to amazon 85.91% vs 76.30% MMD).
- In the 1D KL minimization toy experiment, local linear (LL) estimator reaches KL near zero after 20 iterations, comparable to NW which uses true grad log p.
- Theory: estimation error of local linear slope bounded by K/sqrt(n sigma^d) + O(sigma^2), giving O_p(n^{-2/(d+4)}) with optimal bandwidth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WGF instead of the source baseline for domain adaptation | Office-Caltech-10 · Office-Caltech-10 domain transfers | accuracy | Helped |
| Use WGF instead of JDOT for domain adaptation | Office-Caltech-10 · Office-Caltech-10 domain transfers | accuracy | Helped |
| Use WGF instead of MMD | Office-Caltech-10 · Office-Caltech-10 domain transfers | accuracy | Mixed |
| Use WGF instead of vanilla SVGD with naive gradient descent | Gaussian target N(0, 1) · KL[q_t, p] minimization | KL divergence | Helped |
| Use AdaGrad with SVGD rather than vanilla SVGD with naive gradient descent | Gaussian target N(0, 1) · KL[q_t, p] minimization | KL divergence | Helped |
| Use local linear interpolation rather than estimating the velocity field through a density ratio | WGF velocity estimation · Velocity field estimation | estimation error | Helped |
| Use WGF for missing-value imputation rather than Sinkhorn or GAIN | Missing-value imputation · S-shaped MCAR data | recovery of S-shape | Helped |
| Use WGF for missing-value imputation | Missing-value imputation · Breast Cancer MCAR data | AUROC | No clear effect |
Surprising findings
- To estimate the f-divergence velocity field, one should maximize the variational lower bound of its mirror divergence rather than of the divergence itself.
Limitations
- Requires bounded input domains for the corollaries and overlapping support of p and q (density chasm problem), problematic for high-dimensional data like images.
- Local regression performs poorly in high dimensions; no generator network so sample size must be fixed before the flow.
Experimental setup
Datasets: Office-Caltech-10 UCI Breast Cancer Wisconsin S-shaped toy dataset
Benchmarks: Office-Caltech-10 UCI Breast Cancer Wisconsin S-shape toy imputation
Metrics: classification accuracy AUROC KL divergence estimate (Donsker-Varadhan lower bound)
Baselines: SVGD SVGD with AdaGrad JDOT MMDFlow Sinkhorn GAIN HyperImpute MIWAE
Builds on: Stein Variational Gradient Descent Wasserstein gradient flow of f-divergences JDOT MMDFlow GAIN
Closest papers by meaning
- Flowing Datasets with Wasserstein over Wasserstein Gradient Flows (ICML 2025 · similarity 0.86)
- Wasserstein residuals: Learning Gradient Flows from Population Dynamics (NeurIPS 2026 · similarity 0.86)
- Neural Sampling from Boltzmann Densities: Fisher-Rao Curves in the Wasserstein Geometry (ICLR 2025 · similarity 0.85)
- Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling (NeurIPS 2026 · similarity 0.85)
- Sampling in Unit Time with Kernel Fisher-Rao Flow (ICML 2024 · similarity 0.85)
- Flow matching achieves almost minimax optimal convergence (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.