Research map / Detection and segmentation / Open-set and few-shot detection
Parallel Backpropagation for Shared-Feature Visualization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes parallel backpropagation, a neuron-specific similarity and reweighted gradient saliency method that visualizes features shared between an out-of-category image and a within-category reference image driving macaque body-selective IT neurons.
Problem
Explaining why out-of-category object images strongly activate category-selective (body-selective) neurons in macaque IT cortex, i.e., which visual features are shared between objects and preferred-category images.
Approach
Fit a sparse linear readout on a pretrained adversarially trained ResNet-50 to within-category neural responses, use readout-weighted cosine similarity to pick a reference body image for an object, and reweight per-feature pixel gradients by their shared, neuron-relevant contribution so that saliency highlights shared features and is norm-bounded by the similarity.
Key results
- 93.7% (posterior) and 95.3% (anterior) of channels show significant positive correlation between predicted and recorded responses to object images.
- Visualizations reveal object parts (limbs, tails, heads, torsos) resembling macaque body parts, and some stubby objects activate body-selective channels.
- Integrated-gradient-based saliency yields visualizations almost indistinguishable from vanilla gradients.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train neural response models on body images and test them on object images | Body-trained neural response models · Object images, posterior region | channels with significant positive correlation: 93.7 % | Helped |
| Train neural response models on body images and test them on object images | Body-trained neural response models · Object images, anterior region | channels with significant positive correlation: 95.3 % | Helped |
| Replace vanilla gradients with Integrated Gradients (ablation) | Parallel backpropagation · Shared-feature visualization | visualization similarity to vanilla backpropagation | No clear effect |
Surprising findings
- Strong performance on held-out monkey body images is not a necessary condition for strong generalization to objects.
- Some stubby (non-spiky) objects activate body-selective neurons, contrary to the prior spiky-preference account.
Limitations
- Visualization quality depends on how well the differentiable neuron model captures true tuning properties.
- Visualizations inherit idiosyncrasies of the chosen attribution method.
- Only multi-unit activity from two body patches in two monkeys was analysed.
Experimental setup
Datasets: macaque avatar body images (475 subset of 720) OpenImages object subset plus smaller object sets (6,857 objects) 2068 control body images silhouettes of monkey avatar
Benchmarks: held-out macaque avatar body images object images neural responses (MUA recordings)
Metrics: Pearson correlation between predicted and recorded responses Mann-Whitney U-test neuron-specific similarity
Baselines: Grad-CAM-style similarity saliency integrated gradients
Builds on: Gaussian readout vanilla gradient saliency integrated gradients SmoothGrad
Closest papers by meaning
- In Silico Mapping of Visual Categorical Selectivity Across the Whole Brain (NeurIPS 2025 · similarity 0.86)
- Sparse components distinguish visual pathways & their alignment to neural networks (ICLR 2025 · similarity 0.86)
- Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representations (ICLR 2025 · similarity 0.86)
- Brain Mapping with Dense Features: Grounding Cortical Semantic Selectivity in Natural Images With Vision Transformers (ICLR 2025 · similarity 0.86)
- L-WISE: Boosting Human Visual Category Learning Through Model-Based Image Selection and Enhancement (ICLR 2025 · similarity 0.85)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.