Research map / Probabilistic and scientific modeling / Variational and generative sampling
Identifying Metric Structures of Deep Latent Variable Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that geodesic distances and other Riemannian-geometric quantities of deep latent variable models with injective decoders are identifiable, and validates empirically that they vary less across retrained models than Euclidean latent distances.
Problem
Latent variables of deep latent variable models are not statistically identifiable, so distances and relationships between latent points computed from trained models vary across retraining, making latent-space analysis unreliable for domain experts.
Approach
Instead of identifying latent coordinates, identify relationships between them: connect indeterminacy transformations to charts of the data manifold, so that the pullback Riemannian metric is invariant and geodesic distances, angles, and volumes become identifiable without labels, under mild assumptions (compact latent, injective decoder with full-rank Jacobian, shared image).
Key results
- Across four datasets, geodesic distances showed lower coefficient of variation than Euclidean distances across 30 retrained models (one-sided t-test, all t-statistics between -8.64 and -42.83).
- Geodesic and Euclidean interpolations look visually similar on decoded images, yet their distances differ significantly.
- Proves that Euclidean latent distances are identifiable only if the decoder is flat (zero curvature), so practitioners must choose between cheap flat decoders and curved ones.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use geodesic distances instead of Euclidean distances | 𝓜-flow · MNIST | coefficient of variation: t-statistic -8.64; p-value 1.00 | Helped |
| Use geodesic distances instead of Euclidean distances | VAE · FMNIST | coefficient of variation: t-statistic -16.75; p-value 1.00 | Helped |
| Use geodesic distances instead of Euclidean distances | 𝓜-flow · CIFAR10 | coefficient of variation: t-statistic -42.83; p-value 1.00 | Helped |
| Use geodesic distances instead of Euclidean distances | VAE · CelebA | coefficient of variation: t-statistic -22.33; p-value 1.00 | Helped |
Surprising findings
- Euclidean latent distances are either non-identifiable or require forcing the decoder to be flat, which is a strong constraint on common practice.
Limitations
- Requires injective decoders with full-rank Jacobian, which standard VAEs do not guarantee; the VAE experiment relaxes this assumption.
- Geodesic distances are computationally more expensive than Euclidean distances.
- Depends on the choice of observation-space metric.
Experimental setup
Datasets: MNIST (3-class subset) CIFAR-10 Fashion-MNIST CelebA
Benchmarks: MNIST CIFAR-10 Fashion-MNIST CelebA
Metrics: coefficient of variation of distances across 30 seeds one-sided Student's t-test
Baselines: Euclidean latent distance
Builds on: probabilistic PCA variational autoencoder M-flows indeterminacy transformation identifiability theory
Closest papers by meaning
- Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations (ICML 2025 · similarity 0.88)
- Learning Discrete Latent Variable Structures with Tensor Rank Conditions (NeurIPS 2024 · similarity 0.87)
- Connecting Neural Models Latent Geometries with Relative Geodesic Representations (NeurIPS 2025 · similarity 0.86)
- Latent variable model for high-dimensional point process with structured missingness (ICML 2024 · similarity 0.86)
- Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions (ICLR 2025 · similarity 0.86)
- Mapping the Multiverse of Latent Representations (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.