Research map / Label-efficient and robust learning / Recognition and long-tail
Differentiable Optimization of Similarity Scores Between Models and Brains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Differentiates through similarity measures (CKA, angular Procrustes, NBS, linear/ridge regression) to optimize synthetic data toward neural recordings, showing high scores can miss task-relevant information and that CKA over-weights top principal components.
Problem
Similarity measures such as CKA, linear regression, and Procrustes distance are used to compare models and brains, but it is unclear what drives high scores and what counts as a good score.
Approach
Directly maximize each similarity score by gradient-based optimization of a synthetic dataset initialized from Gaussian noise, track decodability of task variables and per-PC R^2 during optimization, and derive analytically that CKA scales quadratically and NBS linearly with the variance of a perturbed PC.
Key results
- Good similarity score thresholds vary by measure and dataset: for Mante 2013 the scores at which decoding reaches 90% of neural decoding range from under 0.5 to almost 1 across measures; angular Procrustes above 0.5 suffices for Mante but above 0.8 is needed for Siegel 2015.
- Optimized synthetic data under CKA can reach near-maximal similarity while failing to linearly decode task variables, whereas angular Procrustes captures the lower-variance second PC much earlier.
- Theory gives CKA ~ sum over i≠k of (λ_i)^2 / sum of (λ_i)^2 (quadratic in perturbed PC variance) and NBS ~ linear in variance; empirically matched on five neural datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize synthetic data for CKA similarity rather than angular Procrustes similarity | Synthetic datasets matched to neural data · Mante 2013; Siegel 2015 | capture of lower-variance principal components | Hurt |
| Use angular Procrustes instead of CKA when principal-component variances are asymmetric | Synthetic datasets optimized to match the reference data · Two-dimensional synthetic datasets | capture of the lower-variance principal component | Helped |
| Use angular CKA instead of CKA | Synthetic datasets optimized to match neural data · Mante 2013; Siegel 2015; Majaj 2015 | capture of lower-variance principal components | Helped |
| Use CKA or unregularized linear regression similarity to optimize synthetic data | Synthetic datasets matched to neural data · Siegel 2015 | task-variable decoding consistency with neural data | Hurt |
| Use cross-validated and regularized linear regression similarity to optimize synthetic data | Synthetic datasets matched to neural data · Siegel 2015 | task-variable decoding consistency with neural data | Hurt |
| Replace the neural reference with Gaussian data matching its principal-component variances | Principal-component capture during similarity optimization · Neural datasets | similarity score required to reach the principal-component threshold | No clear effect |
| Remove low-variance principal components from the neural dataset (ablation) | Task-variable decoding analysis · Mante 2013 | task-variable decoding accuracy | No clear effect |
| Use CKA rather than NBS to score data with a perturbed principal component | Similarity scoring of a dataset and its perturbed version · Five neural datasets | sensitivity to the variance of the perturbed principal component | Mixed |
| Jointly optimize angular Procrustes and CKA scores | Synthetic datasets · Siegel 2015 | allowable joint similarity scores | Mixed |
| Jointly optimize CKA and Euclidean-RDM RSA using cosine similarity | Synthetic datasets · Similarity-measure relationship analysis | dependence between similarity scores | No clear effect |
| Center Euclidean RDMs before comparing them with cosine similarity (ablation) | RSA · Similarity-measure relationship analysis | equivalence to CKA | Helped |
| Compare RDMs with correlation rather than cosine similarity (ablation) | RSA · Similarity-measure relationship analysis | dependence between RSA and CKA scores | No clear effect |
Surprising findings
- High CKA and even cross-validated, regularized linear regression scores can occur while task-relevant variables are not linearly decodable from the synthetic data.
- Some optimized synthetic datasets decode task variables better than the original neural data, possibly indicating a form of denoising.
Limitations
- Focuses only on differentiable geometry-based similarity measures; other measures not analyzed.
- Results conflate the behavior of gradient-based optimization with the behavior of the similarity measure; local minima not studied.
- Neural data reduced via PCA (99% variance) for some datasets to speed optimization.
Experimental setup
Datasets: Mante 2013 PFC recordings Siegel 2015 FEF recordings Hatsopoulos 2007 M1 recordings Majaj 2015 IT recordings Freeman & Ziemba 2013 V1/V2 recordings
Benchmarks: Mante 2013 Siegel 2015 Hatsopoulos 2007 Majaj 2015 Freeman & Ziemba 2013
Metrics: similarity score decode accuracy R^2 per principal component score to reach PC threshold
Baselines: CKA angular CKA angular Procrustes linear regression ridge regression NBS
Builds on: Centered Kernel Alignment Normalized Bures Similarity Angular Procrustes distance Representational Similarity Analysis
Closest papers by meaning
- Spectral Analysis of Representational Similarity with Limited Neurons (NeurIPS 2025 · similarity 0.89)
- Geometry-Aware Similarity Metrics for Neural Representations on Riemannian and Statistical Manifolds (NeurIPS 2026 · similarity 0.85)
- ReSi: A Comprehensive Benchmark for Representational Similarity Measures (ICLR 2025 · similarity 0.85)
- KoopSTD: Reliable Similarity Analysis between Dynamical Systems via Approximating Koopman Spectrum with Timescale Decoupling (ICML 2025 · similarity 0.84)
- Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.84)
- Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.