Research map / Probabilistic and scientific modeling / Molecules and proteins
Connecting Neural Models Latent Geometries with Relative Geodesic Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes relative geodesic representations that pull back a decoder's output metric to latent space and approximate geodesic distances with straight-line discretized arc lengths, enabling retrieval and zero-shot stitching across autoencoders and vision foundation models.
Problem
Latent space communication: finding transformations between representations learned by distinct neural models on similar data; existing relative representations use cosine similarity, which only captures angle-preserving invariances.
Approach
Treat distinct models as parametrizing the same underlying manifold and build relative representations from approximate geodesic energies/arc lengths of straight latent segments, measured under a pullback metric from a decoder's output space (L2 on classifier logits, or spherical metric from a Diet instance-discrimination head).
Key results
- On CIFAR-10, RelGeo(Diet) reaches average MRR cosine 0.387 vs 0.129 for Rel(Cosine) in retrieval across model pairs; on ImageNet-1k it reaches 0.566 vs 0.221.
- Zero-shot stitching: RelGeo(L2) achieves 0.955 on CIFAR-10 and 0.874 on CIFAR-100 average accuracy, outperforming Rel(Cosine) (0.907 and 0.775).
- Straight-line energy approximation with 8 discretization points reaches Spearman correlation above 0.99 with true geodesic energies on MNIST and CIFAR-10 latents.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RelGeo(Diet) instead of Rel(Cosine) | Vision foundation model retrieval · CIFAR-10, CIFAR-100, ImageNet-1k, CUB, SVHN | MRR cosine: 0.129 ± 0.135 -> 0.387 ± 0.145; 0.166 ± 0.162 -> 0.445 ± 0.142; 0.221 ± 0.178 -> 0.566 ± 0.111; 0.135 ± 0.148 -> 0.523 ± | Helped |
| Use RelGeo(L2) instead of Rel(Cosine) | Vision foundation model retrieval · CIFAR-10, CIFAR-100, ImageNet-1k, CUB, SVHN | MRR cosine: 0.129 ± 0.135 -> 0.047 ± 0.013; 0.166 ± 0.162 -> 0.112 ± 0.031; 0.221 ± 0.178 -> 0.412 ± 0.09; 0.135 ± 0.148 -> 0.28 ± 0 | Mixed |
| Use RelGeo(L2) instead of Rel(Cosine) | Vision foundation model zero-shot stitching · CIFAR-10, CIFAR-100, ImageNet-1k, CUB, SVHN | stitching accuracy: 0.907 ± 0.09 -> 0.955 ± 0.03; 0.775 ± 0.132 -> 0.874 ± 0.055; 0.549 ± 0.152 -> 0.501 ± 0.159; 0.531 ± 0.188 -> 0.595 ± 0 | Mixed |
| Use RelGeo(Diet) instead of Rel(Cosine) | Vision foundation model zero-shot stitching · CIFAR-10, CIFAR-100, ImageNet-1k, CUB, SVHN | stitching accuracy: 0.907 ± 0.09 -> 0.915 ± 0.074; 0.775 ± 0.132 -> 0.775 ± 0.115; 0.549 ± 0.152 -> 0.479 ± 0.17; 0.531 ± 0.188 -> 0.559 ± 0 | Mixed |
| Approximate geodesic energy with straight-line energy | Autoencoder latent representations · MNIST and CIFAR-10 | Spearman rank correlation: 0.99 for MNIST; 1.00 for CIFAR-10 | Helped |
Surprising findings
- RelGeo(L2) gives good stitching accuracy but poor retrieval MRR, while RelGeo(Diet) gives strong MRR but lower accuracy, indicating a trade-off between class-level and instance-level identifiability.
- On the zero-shot stitching table, Rel(Cosine) beats RelGeo(L2) on ImageNet-1k (0.549 vs 0.501) and RelGeo(Diet) is also below Rel(Cosine) on CIFAR-10 accuracy in some settings.
Limitations
- Straight-line approximation degrades in high-curvature regions far from the data support, requiring smaller step sizes.
- Pullback from a classifier or Diet head depends on the decoder's objective; RelGeo(L2) and RelGeo(Diet) trade off class-level and instance-level identifiability.
- Gains over Rel(Cosine) are not uniform; on some stitching settings (e.g., ImageNet-1k) the baseline is competitive.
Experimental setup
Datasets: MNIST FashionMNIST CIFAR-10 CIFAR-100 SVHN CUB ImageNet-1k Flickr30k
Benchmarks: MNIST FashionMNIST CIFAR-10 CIFAR-100 SVHN CUB ImageNet-1k Flickr30k
Metrics: MRR symmetric MRR stitching accuracy Spearman rank correlation
Baselines: Rel(Cosine) linear mapping orthogonal mapping RelGeo(L2)
Builds on: relative representations Diet model stitching stochman
Closest papers by meaning
- Latent Functional Maps: a spectral framework for representation alignment (NeurIPS 2024 · similarity 0.90)
- Geometry-Aware Similarity Metrics for Neural Representations on Riemannian and Statistical Manifolds (NeurIPS 2026 · similarity 0.89)
- Grounding Continuous Representations in Geometry: Equivariant Neural Fields (ICLR 2025 · similarity 0.88)
- Neural Isometries: Taming Transformations for Equivariant ML (NeurIPS 2024 · similarity 0.88)
- From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication (ICLR 2024 · similarity 0.87)
- Estimating Shape Distances on Neural Representations with Limited Samples (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.