Research map / Architectures and efficiency / CNN design
Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a neuro-inspired fixed-weight SubcorticalBlock (retina/LGN model with P/M pathways) in front of the VOneBlock in a ResNet50 to form Early Vision Networks (EVNets), improving V1 alignment, shape bias, and robustness.
Problem
CNNs trained for object recognition remain vulnerable to common corruptions, domain shifts, and adversarial perturbations, and differ from primate vision; prior VOneNets model V1 but omit subcortical (retina/LGN) processing.
Approach
Stack a parameterized SubcorticalBlock (light adaptation, DoG convolution with P and M pathways, contrast normalization, subcortical noise) upstream of the VOneBlock as a fixed-weight multi-stage front-end, with DoG parameters tuned by Bayesian optimization to match LGN response properties.
Key results
- EVResNet50 improves the Robustness Score by 9.3% over ResNet50 (36.1 vs 26.8) and 1.6% over VOneResNet50 (34.5).
- EVResNet50 raises mean V1 response property score to 0.826 vs 0.754 for VOneResNet50 and shape bias to 48.9% vs 31.6%.
- EVResNet50 with PRIME reaches Robustness Score 42.7%, above ResNet50 + PRIME (36.5%) and VOneResNet50 + PRIME (42.1%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SubcorticalBlock before VOneBlock | ResNet50 · Robustness benchmark | Robustness Score: 9.3% | Helped |
| Add SubcorticalBlock before VOneBlock | ResNet50 · V1 response properties | V1 response property score: .754 -> .826 | Helped |
| Add SubcorticalBlock before VOneBlock | ResNet50 · V1 predictivity | V1 predictivity: .375 -> .364 | Hurt |
| Add SubcorticalBlock before VOneBlock | ResNet50 · Shape bias | shape bias: 48.9 | Helped |
| Add SubcorticalBlock before VOneBlock | ResNet50 · ImageNet clean validation | top-1 accuracy: 1.3% | Hurt |
| Add PRIME augmentation to EVResNet50 | EVResNet50 · Robustness benchmark | Robustness Score: 6.2% | Helped |
| Add P-cell pathway (ablation) | EVResNet50 · Robustness benchmark (reduced attack set) | Robustness Score: 38.4 -> 44.6 | Helped |
| Add M-cell pathway (ablation) | EVResNet50 · Robustness benchmark (reduced attack set) | Robustness Score: 44.8 -> 44.6 | No clear effect |
| Add light adaptation (ablation) | EVResNet50 · Robustness benchmark (reduced attack set) | Robustness Score: 44.3 -> 44.6 | No clear effect |
| Add contrast normalization (ablation) | EVResNet50 · Robustness benchmark (reduced attack set) | Robustness Score: 44.4 -> 44.6 | No clear effect |
| Add subcortical noise (ablation) | EVResNet50 · Robustness benchmark (reduced attack set) | Robustness Score: 42.6 -> 44.6 | Helped |
| Add VOneBlock (ablation) | EVResNet50 · ImageNet-C | mean corruption accuracy: 42.7 -> 41.9 | Hurt |
Surprising findings
- Removing the P-cell pathway greatly increased shape bias (77.8%) while reducing robustness, and removing the VOneBlock barely affected overall robustness.
- EVResNet50 has a 1.3% clean accuracy drop relative to the reported ResNet50 accuracy (70.4 vs 75.4 mean), and VOneNets outperform EVNets on V1 predictivity.
Limitations
- Modest reductions in clean accuracy accompany robustness gains (EVResNet50 70.4% vs ResNet50 75.4%).
- Abstracts LGN heterogeneity with only four channels reflecting average spatial response profiles.
- Lower alignment for surround modulation and RF size remains, and SF tuning trades off because the Gabor filter bank was not adapted to the subcortical front-end.
Experimental setup
Datasets: ImageNet-1k ImageNet-C ImageNet-R ImageNet-Cartoon ImageNet-Drawing ImageNet-Sketch Stylized-ImageNet Cue-conflict dataset Brain-Score FreemanZiemba2013 Brain-Score Marques2020
Benchmarks: Brain-Score V1 predictivity Brain-Score V1 response properties shape bias cue-conflict ImageNet-C ImageNet-R ImageNet-Cartoon ImageNet-Drawing ImageNet-Sketch Stylized-ImageNet-16 PGD adversarial attacks
Metrics: top-1 accuracy Robustness Score shape bias V1 predictivity V1 response property score
Baselines: ResNet50 VOneResNet50 AT_Linf PRIME
Builds on: VOneNet VOneBlock PRIME Brain-Score
Closest papers by meaning
- Simulating a Primary Visual Cortex at the Front of CNNs Improves Robustness to Image Perturbations (NeurIPS 2020 · similarity 0.93)
- Towards robust vision by multi-task learning on monkey visual cortex (NeurIPS 2021 · similarity 0.90)
- Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex (NeurIPS 2023 · similarity 0.89)
- When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations (ICLR 2022 · similarity 0.87)
- Harnessing Edge Information for Improved Robustness in Vision Transformers (AAAI 2024 · similarity 0.86)
- SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networks (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.