Research map / Probabilistic and scientific modeling / Variational and generative sampling
Metric Space Magnitude for Evaluating the Diversity of Latent Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes magnitude-based diversity measures (MagArea, MagDiff) that summarise the multi-scale effective size of latent representations, with stability proofs and an efficient Cholesky-based implementation.
Problem
Existing reference-free and reference-based diversity measures for latent representations (AvgSim, GMStds, Vendi Score, recall, coverage) fail basic axioms or depend on fixed-scale neighbourhood parameters, so they misjudge diversity and mode dropping/collapse.
Approach
Use metric space magnitude, a multi-scale invariant measuring the effective number of points, and summarise it as the area under the magnitude function (MagArea) for intrinsic diversity and the integrated difference between two magnitude functions (MagDiff) for reference-based comparison, with an automated convergence-scale selection that removes manual scale parameters.
Key results
- MagArea predicts curvature with 5-fold CV MSE of 0.05 ± 0.03 using piecewise linear regression on a single feature.
- On text diversity prediction, MagArea achieves median rank 1 in R^2 and is best in 77% of resamples, ahead of Vendi Score and AvgSim.
- MagDiff reaches above 90% accuracy (up to 0.99) for identifying the embedding model with 5-NN, outperforming AvgSim, VS and GMStds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MagDiff instead of the strongest single-summary baseline for distinguishing embedding models | 5-NN classifier on text embedding spaces · cnn, patents, arXiv, and bbc; with and without PCA | accuracy | Helped |
| Use MagArea instead of AvgSim to predict text diversity | Text-generation embeddings · Story completion | mean R²: +0.12 | Helped |
| Use MagArea instead of AvgSim to predict text diversity | Text-generation embeddings · Dialogue response and 3-word prompt completion | mean R²: +0.07 | Helped |
| Use MagArea instead of GMStds to predict text diversity | Text-generation embeddings · Three of five embedding models | R² | Helped |
| Use MagDiff instead of recall or coverage | Image embeddings in CIFAR10 mode dropping · Sequential and simultaneous class mode dropping | mode-dropping detection | Helped |
| Use MagDiff instead of recall or coverage | Graph generative models · Mode collapse and mode dropping across graph datasets | rank correlation with perturbation degree | Helped |
| Use MagArea as the feature for piecewise linear rather than quantile regression | Curvature prediction · Unit-disk point clouds with varying curvature | MSE: 0.10 \pm 0.05 -> 0.05 \pm 0.03 | Helped |
Surprising findings
- Recall and coverage respond to sequential versus simultaneous mode dropping differently, wrongly assessing diversity decrease speed; coverage detects simultaneous dropping only after about 70% of points shift.
Limitations
- MagDiff does not measure fidelity and should be used alongside coverage-based metrics.
- Scaling to hundreds of thousands of observations would require novel approximation methods.
- Relies on embedding models whose similarity notions are assumed to be meaningful; alternative task-specific similarities are not investigated.
Experimental setup
Datasets: text generation diversity dataset (1K sets of 10 sentences, story, resp, prompt) HuggingFace text datasets (cnn, patents, arXiv, bbc) CIFAR10 Lobster, Grid, Community, Proteins, Ego graph datasets Circles, Swiss Roll, Gaussian blobs (synthetic)
Benchmarks: curvature regression on point clouds text diversity prediction of decoding temperature embedding model classification (cnn, patents, arXiv, bbc) CIFAR10 mode dropping graph generative model mode collapse and mode dropping
Metrics: R^2 MSE accuracy rank correlation recall coverage
Baselines: AvgSim GMStds Vendi Score recall coverage density precision MMD
Builds on: Vendi Score density and coverage precision and recall
Closest papers by meaning
- Mapping the Multiverse of Latent Representations (ICML 2024 · similarity 0.87)
- Latent Functional Maps: a spectral framework for representation alignment (NeurIPS 2024 · similarity 0.87)
- Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations (ICLR 2025 · similarity 0.86)
- Connecting Neural Models Latent Geometries with Relative Geodesic Representations (NeurIPS 2025 · similarity 0.86)
- Estimating Shape Distances on Neural Representations with Limited Samples (ICLR 2024 · similarity 0.86)
- Geometry-Aware Similarity Metrics for Neural Representations on Riemannian and Statistical Manifolds (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.