Research map / Adversarial robustness and security / Attacks on text and graphs
Causes and Consequences of Representational Similarity in Machine Learning Models
Zeyu Michael Li, Hung Anh Vu, Damilola Awofisayo, Emily Wenger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase dataset overlap between two training sets (ablation) | finetuned ResNet/ViT image classifiers · TinyImageNet | CKA between model pairs: ResNet18 R2=0.99, y=0.02x+0.95; ViT/small R2=0.98 | Improved |
| increase dataset overlap between two training sets (ablation) | ResNet101 · TinyImageNet | CKA between model pairs: R2=0.24 | No clear effect |
| increase dataset overlap between two training sets (ablation) | Llama-3.2-Instruct LoRA CPT · Tiny-Codes | CKA between model pairs: R2=0.01 (1B), 0.03 (3B), 0.02 (11B) | No clear effect |
| increase task and dataset overlap together (ablation) | ResNet/ViT classifiers · TinyImageNet | CKA between model pairs: ResNet18 R2=0.92; ViT/tiny R2=0.86 | Improved |
| increase task and dataset overlap together (ablation) | Llama LoRA CPT · CPT corpora | CKA between model pairs: R2=0.62 (1B), 0.47 (3B), 0.54 (11B) | Mixed |
| increase task overlap with data overlap held at 100% via relabeling (ablation) | ResNet/ViT · ColorShapeDigit800K | CKA between model pairs: ViT/tiny R2=0.84; ResNet18 R2=0.83 | Improved |
| increase label-set task overlap (0%, 50%, 100%) (ablation) | XLM-RoBERTa-base · Multilingual Amazon Reviews | representational similarity | Improved |
| compare overlap type by mutual information with similarity (ablation) | model families · CKA/CKNNA | mutual information: ResNet dataset 1.303 vs task+dataset 0.734; nanoGPT 0.198 vs 1.052 | Mixed |
| increase dataset/task overlap between model pairs (ablation) | ViT classifiers, MI-FGSM transfer attack · TinyImageNet | transfer attack success rate: ViT/tiny R2=0.83, y=0.20x+0.57 | Improved |
| increase dataset/task overlap between LLM pairs (ablation) | Llama 3.2 continued-pretrained models · jailbreak transfer, Llama-70B judge | transfer jailbreak ASR: R2=0.01, 0.00, 0.04 | No clear effect |
| train longer on LLM continued pretraining to test insufficient adaptation (ablation) | Llama CPT and instruction finetuning · perplexity across overlap settings | perplexity | Improved |
Closest papers by meaning
- Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.90)
- When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective (NeurIPS 2025 · similarity 0.88)
- Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity (ICLR 2025 · similarity 0.87)
- Representational Similarity via Interpretable Visual Concepts (ICLR 2025 · similarity 0.87)
- Representational Difference Explanations (NeurIPS 2025 · similarity 0.86)
- ReSi: A Comprehensive Benchmark for Representational Similarity Measures (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
