Research map / Large language models / AI and society
Scale Alone Does not Improve Mechanistic Interpretability in Vision Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale model size or training dataset size | The nine evaluated vision models · ImageNet Mechanistic Interpretability evaluation | per-unit interpretability | No clear effect |
| Use latest-generation ConvNeXt or ViT architectures instead of older architectures | Vision models · ImageNet Mechanistic Interpretability evaluation | per-unit interpretability | Hurt |
| Use feature visualizations instead of natural exemplars as explanations | Evaluated vision models · ImageNet Mechanistic Interpretability evaluation | human interpretability score | Hurt |
| Train ResNet-50 for adversarial robustness | ResNet-50 · ImageNet Mechanistic Interpretability evaluation, synthetic explanations | interpretability score | Helped |
| Train ResNet-50 for adversarial robustness | ResNet-50 · ImageNet Mechanistic Interpretability evaluation, natural exemplars | interpretability score | No clear effect |
| Make query images less extreme, sampling from lower activation percentiles | ResNet-50 and CLIP ResNet-50 · ImageNet Mechanistic Interpretability evaluation, natural exemplars | human accuracy | Hurt |
| Make query images less extreme, sampling from lower activation percentiles | ResNet-50 and CLIP ResNet-50 · ImageNet Mechanistic Interpretability evaluation, synthetic explanations | human accuracy | Hurt |
| Increase classification performance | Vision models · ImageNet Mechanistic Interpretability evaluation, synthetic explanations | interpretability | Hurt |
| Increase classification performance | Vision models · ImageNet Mechanistic Interpretability evaluation, natural exemplars | interpretability | No clear effect |
| Pre-train on large-scale datasets | CLIP ResNet-50 and CLIP ViT · ImageNet Mechanistic Interpretability evaluation, natural exemplars | layer-wise interpretability across network depth | Mixed |
| Use local contrast of activation maps to predict unit interpretability | Sampled convolutional-network units · ImageNet Mechanistic Interpretability evaluation | association with interpretability: -0.24 ≤ ρ ≤ 0.14 | No clear effect |
| Use pixel-wise activation sparseness to predict unit interpretability | Sampled vision-model units · ImageNet Mechanistic Interpretability evaluation | association with interpretability | No clear effect |
Closest papers by meaning
- Measuring Per-Unit Interpretability at Scale Without Humans (NeurIPS 2024 · similarity 0.91)
- MICLIP: Learning to Interpret Representation in Vision Models (ICLR 2026 · similarity 0.88)
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.87)
- Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.87)
- Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Stream (ICML 2025 · similarity 0.86)
- Enhancing Pre-trained Representation Classifiability can Boost its Interpretability (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.