Research map / Probabilistic and scientific modeling / Variational and generative sampling
Learning Distances from Data with Normalizing Flows and Score Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns density-based (Fermat) distances by using normalizing flows for more accurate graph edge weights and sliced score matching for a score-guided relaxation that computes smooth geodesics, evaluated against ground-truth geodesics on 2D GMMs/spirals and high-dimensional standard normals.
Problem
Fermat distances (density-based distances) are principled metrics for data manifolds, but existing kNN-graph estimators converge very slowly due to poor density estimates and produce rough geodesics that degrade in high dimensions.
Approach
Replace nearest-neighbor density estimates in the kNN graph with normalizing-flow densities, refine paths via a relaxation solver of the geodesic ODE (reparameterized to constant Euclidean speed) using a sliced-score-matching score model, and propose a dimension-scaled temperature beta=1/D.
Key results
- Density-weighted graphs using normalizing-flow densities converge to ground-truth distances at nearly the same rate as ground-truth-density graphs, far faster than the power-weighted kNN method.
- Graph-based methods degrade rapidly as dimension increases up to 25D, while score-based relaxation maintains good LPR across dimensions (mean LPR at n=200,000).
- Proposed dimension-scaled temperature beta=1/D yields more consistent geodesics across dimensions than beta=1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace nearest-neighbor density estimates with a learned normalizing flow for graph edge weights | Density-weighted graph method · Five two-dimensional datasets | Log Path Ratio | Helped |
| Use score matching to estimate the score directly instead of differentiating a normalizing flow | Relaxation · Standard normal up to 25 dimensions | Log Path Ratio | Helped |
| Scale the Fermat metric with dimension using β=1/D rather than β=1 | Fermat distance · Standard normal across dimensions | Log Path Ratio | Helped |
| Replace learned density estimates with ground-truth density estimates for graph edge weights (ablation) | Density-weighted graph method · Five two-dimensional datasets | Log Path Ratio | Helped |
| Use inverse-of-mean nearest-neighbor density for graph edge weights (ablation) | Power-weighted graph method · Two-dimensional Gaussian mixture dataset | Log Path Ratio | Hurt |
| Use mean-of-inverse nearest-neighbor density for graph edge weights (ablation) | Power-weighted graph method · Two-dimensional Gaussian mixture dataset | Log Path Ratio | Hurt |
| Use maximum endpoint nearest-neighbor density for graph edge weights (ablation) | Power-weighted graph method · Two-dimensional Gaussian mixture dataset | Log Path Ratio | Hurt |
| Use minimum endpoint nearest-neighbor density for graph edge weights (ablation) | Power-weighted graph method · Two-dimensional Gaussian mixture dataset | Log Path Ratio | Hurt |
| Use graph-based paths without score-based relaxation (ablation) | High-dimensional geodesic computation · Standard normal across dimensions | Log Path Ratio | Hurt |
| Use learned-score relaxation instead of graph-based paths alone (ablation) | High-dimensional geodesic computation · Standard normal across dimensions | Log Path Ratio | Helped |
Surprising findings
- Geodesic scores derived by differentiating a normalizing flow were too noisy, so a separately trained sliced-score-matching model worked better for relaxation.
- Existing power-weighted kNN Fermat distance estimators converge extremely slowly even on a 2D standard normal, with density estimation identified as the bottleneck.
Limitations
- Evaluation limited to toy 2D distributions and Gaussian-like high-dimensional cases with known densities; real-world data not validated beyond preliminary MNIST latent experiments.
- Graph-based initialization and relaxation rely on learned models whose accuracy in high dimensions is not established.
- Theoretical convergence behaviour of the proposed methods is left to future work.
Experimental setup
Datasets: synthetic 2D standard normal 2D Gaussian mixture (3 components) circle single spiral two interleaved spirals high-dimensional standard normal up to 25D MNIST
Benchmarks: 5 two-dimensional datasets with known ground-truth densities high-dimensional standard normal (1-25D) MNIST latent-space geodesics
Metrics: Log Path Ratio (LPR) mean/std of log distances
Baselines: power-weighted shortest path (kNN graph) density-weighted graph with ground-truth density graph-based with scaled metric
Builds on: Fermat distances power-weighted shortest path distances sliced score matching FrEIA normalizing flows
Closest papers by meaning
- Estimating Riemannian Metric with Noise-Contaminated Intrinsic Distance (NeurIPS 2023 · similarity 0.89)
- Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models (NeurIPS 2025 · similarity 0.88)
- Curvilinear Distance Metric Learning (NeurIPS 2019 · similarity 0.87)
- Riemannian Metric Matching for Scalable Geometric Modeling of Distributions (ICML 2026 · similarity 0.87)
- Finsler Multi-Dimensional Scaling: Manifold Learning for Asymmetric Dimensionality Reduction and Embedding (CVPR 2025 · similarity 0.86)
- Volume-Aware Distance for Robust Similarity Learning (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.