Research map / Vision-language and multimodal / Human and embodied modeling
CoSy: Evaluating Textual Explanations of Neurons
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoSy, an architecture-agnostic framework that scores textual neuron explanations by generating synthetic images from the explanation with a text-to-image model and comparing neuron activations against a control dataset (AUC and MAD), then benchmarks MILAN, INVERT, CLIP-Dissect and FALCON.
Problem
Textual explanations of neurons in vision models lack a unified quantitative evaluation measure, so methods cannot be compared across models and explanation approaches.
Approach
Synthesize images from each open-vocabulary explanation with a text-to-image model, then score the explanation by how well the neuron's activations separate these synthetic images from a natural control dataset, using AUC and a mean activation difference (MAD).
Key results
- On ImageNet ResNet50 avgpool, INVERT achieved AUC 0.96 and CLIP-Dissect MAD 4.98, while MILAN scored AUC 0.44 and MAD -0.08.
- True ground-truth explanations scored AUC 0.98-0.99 versus 0.44-0.49 for random explanations across four ImageNet models.
- Across layers of ResNet18, explanation quality generally declines in lower layers; MILAN and FALCON stay near AUC 0.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ground-truth class explanations instead of randomly selected explanations | CoSy evaluation of output neurons · ImageNet output neurons, four models | AUC | Helped |
| Use ground-truth class explanations instead of randomly selected explanations | CoSy evaluation of output neurons · ImageNet output neurons, four models | MAD | Helped |
| Use INVERT rather than MILAN | Neuron descriptions evaluated with CoSy · ImageNet and Places365 latent layers | AUC | Helped |
| Use CLIP-Dissect rather than MILAN | Neuron descriptions evaluated with CoSy · ImageNet and Places365 latent layers | MAD | Helped |
| Use MILAN explanations | Neuron descriptions evaluated with CoSy · ImageNet and Places365 latent layers | AUC | Hurt |
| Use CLIP-Dissect rather than INVERT | Neuron descriptions evaluated with CoSy · ImageNet ResNet18 Avgpool | AUC: 0.93±0.11 vs 0.93±0.11 | No clear effect |
| Use more detailed Prompt 5 with SDXL rather than other tested prompts | Synthetic image generation for ImageNet concepts · ImageNet | Cosine Similarity: 0.72±0.08 | Helped |
| Use Prompt 3 rather than other tested prompts | Synthetic image generation for Places365 concepts · Places365 | Cosine Similarity: 0.64±0.09 | Helped |
| Use SDXL rather than Stable Cascade | Synthetic image generation · ImageNet and Places365 | Cosine Similarity | Helped |
| Exclude the target class from the control dataset | CoSy sanity-check evaluation · ImageNet output neurons | AUC | No clear effect |
| Evaluate explanations for later rather than lower ResNet18 layers | Neuron description methods · ImageNet ResNet18 layers 1–4 and output layer | AUC | Helped |
Surprising findings
- MILAN and FALCON explanations in lower and some higher layers yield activation distributions overlapping the control data, i.e., near-random explanations.
- CLIP-Dissect outperforms INVERT on MAD for convolutional networks even though INVERT leads on AUC, which is its optimization target.
Limitations
- Relies on text-to-image models that may not cover certain concepts, and abstract concepts are hard to generate.
- INVERT may be biased toward AUC in evaluation since it optimizes AUC.
- Evaluation limited to computer vision models and explanation quality as defined by AUC and MAD.
Experimental setup
Datasets: ImageNet Places365 Broden
Benchmarks: ImageNet Places365
Metrics: AUC MAD Cosine Similarity LPIPS Euclidean Distance
Baselines: MILAN INVERT CLIP-Dissect FALCON
Builds on: INVERT CLIP-Dissect MILAN FALCON CompExp NetDissect
Closest papers by meaning
- Automated Natural Language Explanation of Deep Visual Neurons with Large Models (Student Abstract) (AAAI 2024 · similarity 0.88)
- Evaluating Neuron Explanations: A Unified Framework with Sanity Checks (ICML 2025 · similarity 0.87)
- LINE: LLM-based Iterative Neuron Explanations for Vision Models (NeurIPS 2026 · similarity 0.85)
- WWW: A Unified Framework for Explaining What Where and Why of Neural Networks by Interpretation of Neuron Concepts (CVPR 2024 · similarity 0.85)
- Explanation Bottleneck Models (AAAI 2025 · similarity 0.85)
- FaCT: Faithful Concept Traces for Explaining Neural Network Decisions (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.