Research map / Vision-language and multimodal / Human and embodied modeling
Automated Natural Language Explanation of Deep Visual Neurons with Large Models (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a post-hoc, training-free framework that uses an LLM (GPT-3) to build concept vocabularies and CLIP to match them to top-activating image patches, producing natural-language explanations of deep vision neurons without human annotation.
Problem
Neuron-level explanations of deep vision networks currently depend on human annotators to describe what neurons detect, which does not scale to large models.
Approach
Collect top-activating image patches for a neuron, have an LLM generate a domain-specific concept vocabulary, and align patches with concepts using an off-the-shelf VLM (CLIP) so that no dataset construction or extra training is needed.
Key results
- Generated neuron explanations qualitatively agree with human annotations across AlexNet, ResNet50 and ViT neurons (e.g., unit 559 in ResNet50-Places layer4 described as lighthouse-related).
- Ablating a single unit (ResNet50-ImageNet layer4-900, 'a dip or spread made from mashed avocado') causes an 84% accuracy drop on the guacamole category.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the unit representing mashed avocado dip or spread (ablation) | ResNet50-ImageNet · ImageNet guacamole category | test accuracy: 84% | Helped |
Limitations
- Evaluation is mostly qualitative with a small number of example neurons.
- Relies on the quality of LLM-generated vocabulary and CLIP alignment.
Experimental setup
Datasets: ImageNet Places365
Benchmarks: ImageNet Places365
Metrics: category test accuracy drop after neuron ablation qualitative agreement with human annotations
Baselines: human annotations
Builds on: CLIP Network Dissection
Closest papers by meaning
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.88)
- CoSy: Evaluating Textual Explanations of Neurons (NeurIPS 2024 · similarity 0.88)
- Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.87)
- LINE: LLM-based Iterative Neuron Explanations for Vision Models (NeurIPS 2026 · similarity 0.87)
- NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning (NeurIPS 2026 · similarity 0.86)
- Knowledge-Aware Neuron Interpretation for Scene Classification (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.