Research map / Vision-language and multimodal / Human and embodied modeling

LINE: LLM-based Iterative Neuron Explanations for Vision Models

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LINE, a training-free, black-box iterative LLM + text-to-image loop that proposes and scores open-vocabulary concepts to label vision-model neurons, outperforming CLIP-Dissect, INVERT and DnD on CoSy.

Problem

Automatic neuron labeling in vision models is limited by predefined concept vocabularies and overly specific descriptions that miss higher-order global concepts.

Approach

An LLM iteratively proposes new concepts from a scoreboard of previously scored concepts (initialized from ImageNet labels); a T2I model synthesizes images for each concept, the neuron's mean activation scores them, and a final summary step generalizes the top concepts into a higher-order label.

Key results

• On CoSy, LINE reaches AUC 0.96 on ResNet50 ImageNet versus 0.87 for CLIP-Dissect and 0.88 for INVERT, and 0.94 on ResNet50 Places365 versus 0.89 for CLIP-Dissect.
• Final labels are generated (not predefined) for 16-36% of neurons across the evaluated models and layers.
• On ResNet50, swapping SDXL for FLUX.1 [dev] gives slightly higher activations, but the difference is not statistically significant, so SDXL is used for lower cost.

Surprising findings

• Crop-based DnD performs surprisingly poorly on CoSy, which the authors attribute to cropping producing uninformative samples.
• Swapping the T2I model changes the top concept in 71% of cases, though the resulting labels remain semantically similar.

Limitations

• Initialization is constrained by the imprecise predefined ImageNet vocabulary, and expanding it adds computational overhead.
• For polysemantic neurons the loop may converge on a single concept.
• Relies on LLM and T2I models that may lack fidelity in specialized domains such as medical or satellite imaging.

Experimental setup

Datasets: ImageNet-1K Places365 CoSy synthetic concept images Salient ImageNet

Benchmarks: CoSy Salient ImageNet

Metrics: AUC MAD

Baselines: CLIP-Dissect INVERT Describe-and-Dissect DiffExplainer DEXTER FALCON MILAN

Builds on: Describe-and-Dissect MILS CoSy CLIP-Dissect INVERT

Closest papers by meaning

• NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning (NeurIPS 2026 · similarity 0.90)
• Knowledge-Aware Neuron Interpretation for Scene Classification (AAAI 2024 · similarity 0.88)
• Concept-RuleNet: Grounded Multi-Agent Neurosymbolic Reasoning in Vision Language Models (AAAI 2026 · similarity 0.88)
• Automated Natural Language Explanation of Deep Visual Neurons with Large Models (Student Abstract) (AAAI 2024 · similarity 0.87)
• VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set (NeurIPS 2025 · similarity 0.86)
• Show and Tell: Visually Explainable Deep Neural Nets via Spatially-Aware Concept Bottleneck Models (CVPR 2025 · similarity 0.86)