Research map / Probabilistic and scientific modeling / Variational and generative sampling
Injective flows for star-like manifolds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes injective normalizing flows for star-like manifolds whose Jacobian determinant is computed exactly in O(d^2), and applies them to variational inference on penalized-likelihood norm-ball manifolds and probability simplex posteriors for mixing models.
Problem
Density estimation on manifolds with injective flows requires an expensive, typically approximated Jacobian determinant, which hurts variational inference where only an unnormalized target is available.
Approach
Parametrize star-like manifolds with generalized spherical coordinates (angles plus a radius function r(theta)) and compose a bijective flow on angles, a radius-padding injective map, and a spherical-to-Cartesian diffeomorphism; this yields an exact Jacobian determinant computable in O(d^2) via a pseudo-determinant/adjugate identity.
Key results
- Exact Jacobian computation scales as approximately O(d^2.96) for the explicit method versus O(d^1.81) for the proposed approach on hypersphere.
- On 3D targets (von Mises-Fisher, mixture of 50 vMF, sinusoidal on deformed sphere) the learned density reaches MSE 0.013 and 0.011 on the first two.
- Training with the Hutchinson approximate Jacobian yields poor density reconstruction compared with the exact Jacobian model on an lp-pseudo-norm ball with p=0.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the exact Jacobian instead of the Hutchinson trace estimator | Injective flow for star-like manifolds · Uniform distribution on an l_p-(pseudo) norm ball with p=0.5 | density reconstruction and sample quality | Helped |
| Use the efficient exact Jacobian computation instead of explicit Jacobian computation | Injective flow for star-like manifolds · Hypersphere, Jacobian determinant computation | runtime scaling: O(d^{2.96}) -> O(d^{1.81}) | Helped |
Surprising findings
- Approximate (Hutchinson) Jacobian training fails to match exact-Jacobian training even when using up to 100 Gaussian samples, suggesting approximation error matters strongly in variational inference.
Limitations
- Only applies to star-like manifolds of codimension one (intrinsic dimension d-1); not directly generalizable to lower-dimensional manifolds.
- Expressivity depends on the chosen bijective layers and can be limited in modeling many modes.
Experimental setup
Datasets: synthetic regression data (Wishart design) stock returns dataset (10 stocks, 200 time steps)
Metrics: runtime scaling exponent MSE of learned log-density MSE between modeled and ground-truth density
Baselines: Hutchinson trace estimator injective flow MCMC sampler (conjugate Dirichlet-multinomial) subjective Bayes Laplace prior normalizing flow
Builds on: normalizing flows circular splines FlowConductor
Closest papers by meaning
- Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows (NeurIPS 2021 · similarity 0.88)
- Learning Distributions on Manifolds with Free-Form Flows (NeurIPS 2024 · similarity 0.87)
- Lifting Architectural Constraints of Injective Flows (ICLR 2024 · similarity 0.86)
- Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes (ICML 2024 · similarity 0.86)
- Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models (NeurIPS 2024 · similarity 0.85)
- Categorical Flow Matching on Statistical Manifolds (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.