Research map / Label-efficient and robust learning / Recognition and long-tail
Representational Difference Explanations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Representational Difference Explanations (RDX), a training-free method that builds a locally biased difference affinity matrix from two models' normalized neighborhood distances and spectrally clusters it to surface concept grids unique to one representation, evaluated with a new binary success rate metric.
Problem
Existing dictionary-learning XAI methods (SAE, NMF, KMeans, PCA) produce explanations unrelated to known differences when used to compare two learned representations, especially when representational differences are subtle.
Approach
Instead of explaining each representation separately, RDX constructs a difference affinity matrix that emphasizes pairs of inputs close in one representation but far in the other, clusters it spectrally, and defines each concept directly as a grid of its member images, so concepts and explanations coincide.
Key results
- RDX achieves higher binary success rate than all DL baselines on known-difference experiments (MNIST modifications, CUB PCBM concept removal) and on unknown-difference comparisons (DINO vs DINOv2, CLIP vs CLIP-iNat), with and without alignment.
- On CUB, RDX explanations isolate the spotted-wing concept after removing it from the PCBM, while CNMF explanations are unrelated and indistinguishable between models.
- Baseline methods (NMF/CNMF, SAE, KMeans) often produce indistinguishable explanations for both representations when representations are similar, and alignment makes baseline discovery harder while RDX remains robust.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DINOv2 instead of DINO | DINO representation · Mittens | linear probe accuracy: 0.933 -> 0.965 | Helped |
| Use DINOv2 instead of DINO | DINO representation · Primates | linear probe accuracy: 0.918 -> 0.921 | Helped |
| Fine-tune CLIP on iNaturalist data | CLIP representation · Maples | linear probe accuracy: 0.752 -> 0.796 | Helped |
| Fine-tune CLIP on iNaturalist data | CLIP representation · Corvids | linear probe accuracy: 0.790 -> 0.787 | Hurt |
| Use RDX instead of dictionary-learning baselines | Model representation comparisons · MNIST and CUB PCBM known-difference experiments | BSR | Helped |
| Use RDX instead of baseline methods | Large vision model comparisons · ImageNet and iNaturalist class subsets | BSR | Helped |
Surprising findings
- Standard NMF with maximum sampling produces effectively the same explanation grid for both representations on the MNIST 3/5/8 checkpoints, despite a 3% accuracy gap.
- SAE explanations for the stronger model appear less mixed than for the expert model, which the authors attribute to random variation in SAE concepts.
Limitations
- Computing the full pairwise distance matrix requires O(n^2) memory, limiting scalability to large n (tested up to ~5000 points).
- BSR is a proxy metric and can be satisfied by explanations that differ in human-interpretable focus; qualitative inspection is needed.
- Relies on Euclidean distances, which may be misleading on curved data manifolds; focused on vision models only.
Experimental setup
Datasets: MNIST CUB ImageNet iNaturalist
Benchmarks: MNIST modified-label comparisons CUB PCBM concept removal ImageNet subsets (Mittens, Primates) iNaturalist subsets (Maples, Corvids)
Metrics: binary success rate (BSR) SemanticLens redundancy clarity polysemanticity
Baselines: TKSAE SAE NMF CNMF PCA KMeans NLMCD
Builds on: sparse autoencoders non-negative matrix factorization centered kernel alignment SemanticLens spectral clustering
Closest papers by meaning
- Representational Similarity via Interpretable Visual Concepts (ICLR 2025 · similarity 0.88)
- Objective drives the consistency of representational similarity across datasets (ICML 2025 · similarity 0.87)
- Local vs. Global Interpretability: A Computational Complexity Perspective (ICML 2024 · similarity 0.86)
- I-CEE: Tailoring Explanations of Image Classification Models to User Expertise (AAAI 2024 · similarity 0.85)
- CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantification (CVPR 2025 · similarity 0.85)
- When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.