Research map / Label-efficient and robust learning / Recognition and long-tail

Geometry-Aware Similarity Metrics for Neural Representations on Riemannian and Statistical Manifolds

NeurIPS 2026 · Label-efficient and robust learning › Recognition and long-tail · approach: Correspondence and causal representation learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces metric similarity analysis (MSA), which compares the intrinsic Riemannian (pullback) geometry of neural representations via a bounded spectral-ratio distance over SPD matrices, integrated over the input manifold.

Problem

Existing similarity measures (CCA, RSA, CKA, Procrustes) compare extrinsic state-space geometry of representations, which may miss differences in intrinsic geometry between fundamentally different neural network solutions.

Approach

Compare networks by the pullback Riemannian metrics they induce on the input manifold, using a new bounded spectral-ratio distance between SPD matrices integrated over the manifold; the resulting MSA is invariant to state-space rotations and local coordinate changes.

Key results

• Procrustes reports near-perfect similarity between rich and lazy networks, whereas MSA reports low cross-regime similarity relative to within-regime similarity across seeds.
• MSA recovers a hierarchy of similarities that scales with the number of differing hyperparameters (task, learning regime, seed), while RSA cannot distinguish rich, lazy and untrained models.
• On the working-memory task, MSA separates RNN and SSM by architecture and training stage, whereas RSA ignores training stage and DSA is even less sensitive.

Surprising findings

• Procrustes and RSA report high similarity between rich and lazy networks despite distinct intrinsic geometries.
• RSA and CCA give intermediate similarity for fundamentally different computations (trained vs untrained models), while MSA gives zero similarity.

Limitations

• Requires an explicit characterization of the input data manifold, which is hard for data such as neuroscience recordings without a known manifold.
• Does not capture how representations are used by downstream layers or decoders.
• Results are correlational rather than causal.

Experimental setup

Datasets: synthetic 2D manifold angle-classification inputs synthetic angle-sequence working memory inputs on a torus

Metrics: MSA similarity RSA similarity CCA similarity Procrustes similarity DSA similarity

Baselines: RSA CCA Procrustes DSA CKA

Builds on: Procrustes analysis representational similarity analysis canonical correlation analysis dynamical similarity analysis affine-invariant Riemannian metric

Closest papers by meaning

• Estimating Shape Distances on Neural Representations with Limited Samples (ICLR 2024 · similarity 0.89)
• Connecting Neural Models Latent Geometries with Relative Geodesic Representations (NeurIPS 2025 · similarity 0.89)
• RNNs perform task computations by dynamically warping neural representations (NeurIPS 2025 · similarity 0.88)
• Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.87)
• Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations (ICLR 2025 · similarity 0.87)
• Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces (NeurIPS 2026 · similarity 0.87)