Research map / Architectures and efficiency / Attention and transformers
Discovering Influential Neuron Path in Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neuron Path, a joint integrated-gradient attribution (JAS) with layer-progressive greedy search to find influential FFN neuron paths across layers in vision Transformers, and analyzes class-level patterns and pruning.
Problem
Existing neuron attribution methods for vision Transformers score neurons individually and ignore the joint influence and information flow of neurons across layers, leaving the inner mechanism of ViTs opaque.
Approach
Defines a joint attribution score (JAS) that integrates gradients over a set of neurons across layers, and greedily selects one neuron per layer to maximize JAS, yielding a neuron path from input to output; the path is then used for interventions, class-level analysis, and pruning.
Key results
- Neuron Path JAS on ViT-B-16 is 0.4078 vs 0.0412 for Influence Pattern and -0.0034 for Activation.
- Removing neurons selected by Neuron Path lowers accuracy by 2.40% (ViT-B-16) to 26.50% (MAE-B-16) versus Activation's near-zero or mild changes.
- Pruning with top-5 JAS neurons per layer and 100% masking retains 0.5407 accuracy on ViT-B-16 at depth 12, versus 0.5320 for the activation baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use joint attribution to locate the neuron path instead of Influence Pattern | ViT-B-16 · ImageNet1k validation | Joint Attribution Score: 0.0412 -> 0.4078 | Helped |
| Use joint attribution to locate the neuron path instead of Influence Pattern | ViT-B-32 · ImageNet1k validation | Joint Attribution Score: 0.0841 -> 0.6610 | Helped |
| Use joint attribution to locate the neuron path instead of Influence Pattern | ViT-L-32 · ImageNet1k validation | Joint Attribution Score: 0.1227 -> 1.0086 | Helped |
| Use joint attribution to locate the neuron path instead of Influence Pattern | MAE-B-16 · ImageNet1k validation | Joint Attribution Score: 0.0030 -> 0.0095 | Helped |
| Intervene on neurons selected by Neuron Path rather than Influence Pattern by removing them | ViT-B-16 · ImageNet1k validation | Removal Accuracy Deviation: -0.50% -> -2.40% | Hurt |
| Intervene on neurons selected by Neuron Path rather than Influence Pattern by removing them | ViT-B-32 · ImageNet1k validation | Removal Accuracy Deviation: -1.24% -> -3.81% | Hurt |
| Intervene on neurons selected by Neuron Path rather than Influence Pattern by removing them | ViT-L-32 · ImageNet1k validation | Removal Accuracy Deviation: -1.41% -> -5.28% | Hurt |
| Intervene on neurons selected by Neuron Path rather than Influence Pattern by removing them | MAE-B-16 · ImageNet1k validation | Removal Accuracy Deviation: -15.67% -> -26.50% | Hurt |
| Double the values of neurons selected by Neuron Path rather than Influence Pattern | ViT-B-16 · ImageNet1k validation | Enhancement Accuracy Deviation: 0.46% -> 2.04% | Helped |
| Double the values of neurons selected by Neuron Path rather than Influence Pattern | ViT-B-32 · ImageNet1k validation | Enhancement Accuracy Deviation: 0.83% -> 3.06% | Helped |
| Double the values of neurons selected by Neuron Path rather than Influence Pattern | ViT-L-32 · ImageNet1k validation | Enhancement Accuracy Deviation: 1.12% -> 5.02% | Helped |
| Double the values of neurons selected by Neuron Path rather than Influence Pattern | MAE-B-16 · ImageNet1k validation | Enhancement Accuracy Deviation: 4.15% -> 7.28% | Helped |
Surprising findings
- Supervised ViT-B-16 and self-supervised MAE-B-16 show almost inverted distributions of knowledge neurons across layers.
- Retaining only about five neurons per layer on the FFN paths keeps classification accuracy around 52-54% even when all other neurons are zeroed, suggesting high redundancy.
Limitations
- Only neurons in the FFN first linear layer are analyzed, not the full Transformer block.
- Evaluated only on discriminative image classification with ImageNet1k validation data.
Experimental setup
Datasets: ImageNet1k validation set
Benchmarks: ImageNet1k
Metrics: Joint Attribution Score Removal Accuracy Deviation Enhancement Accuracy Deviation probability deviation pruning accuracy
Baselines: Activation Influence Pattern ViT-Slim
Builds on: knowledge attribution integrated gradients pattern influence
Closest papers by meaning
- Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization (ICLR 2024 · similarity 0.85)
- Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Data (ICLR 2024 · similarity 0.85)
- LINE: LLM-based Iterative Neuron Explanations for Vision Models (NeurIPS 2026 · similarity 0.85)
- Interpreting the Second-Order Effects of Neurons in CLIP (ICLR 2025 · similarity 0.85)
- Knowledge-Aware Neuron Interpretation for Scene Classification (AAAI 2024 · similarity 0.85)
- Transformer brain encoders explain human high-level visual responses (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.