Research map / Label-efficient and robust learning / Recognition and long-tail
Reliability of CKA as a Similarity Measure in Deep Learning
MohammadReza Davari, Stefan Horoi, Amine Natik, Guillaume Lajoie, Guy Wolf, Eugene Belilovsky
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Translate a subset of representations from one cube in a random direction | Artificial representations · Artificial subset translation | linear CKA: drops below 0.2 | Hurt |
| Translate a subset of representations from one cube in a random direction | Artificial representations · Artificial subset translation | RBF CKA: drops below 0.2 | Hurt |
| Translate one class of representations in a direction that preserves the separating hyperplanes | 9-layer CNN representations · CIFAR10 training representations | linear CKA: quickly drop to 0 | Hurt |
| Translate a single representation as an outlier | 9-layer CNN representations · CIFAR10 training representations | linear CKA | Hurt |
| Translate a subset while preserving the separating hyperplanes | 9-layer CNN representations · CIFAR10 training representations | classification accuracy: >90% accuracy on both sets | No clear effect |
| Optimize the CKA map while preserving outputs with distillation | Trained CNN · CIFAR10 | model performance | No clear effect |
| Optimize the CKA map to increase similarity between the first and last layers | Trained CNN · CIFAR10 | model performance | No clear effect |
| Optimize the CKA map to increase similarity among all layers | Trained CNN · CIFAR10 | model performance | No clear effect |
| Optimize the CKA map to decrease similarity among all layers | Trained CNN · CIFAR10 | model performance | No clear effect |
| Optimize wide-network CKA maps to match a thin network's target map | ResNet-34 · CIFAR10 | model performance | No clear effect |
| Replace distillation loss with cross-entropy loss (ablation) | CKA map optimization · CIFAR10 | CKA map manipulation | No clear effect |
| Use CKA-map optimization on a Vision Transformer | Vision Transformer · CIFAR10 | model performance | No clear effect |
Closest papers by meaning
- Enhancing Diversity in Bayesian Deep Learning via Hyperspherical Energy Minimization of CKA (NeurIPS 2024 · similarity 0.89)
- Differentiable Optimization of Similarity Scores Between Models and Brains (ICLR 2025 · similarity 0.88)
- Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity (ICLR 2025 · similarity 0.87)
- Spectral Analysis of Representational Similarity with Limited Neurons (NeurIPS 2025 · similarity 0.85)
- Geometry-Aware Similarity Metrics for Neural Representations on Riemannian and Statistical Manifolds (NeurIPS 2026 · similarity 0.84)
- ReSi: A Comprehensive Benchmark for Representational Similarity Measures (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
