Research map / Architectures and efficiency / Spiking networks
Long-Range Feedback Spiking Network Captures Dynamic and Static Representations of the Visual Cortex under Movie Stimuli
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LoRaFB-SNet, a deep spiking network with long-range top-down feedback, and TSRSA, a time-series RSA metric, to show it better matches mouse visual cortex responses to movie stimuli than CORnet, ResNet-2p-CPC and SEW-ResNet.
Problem
Existing DNNs model visual cortex responses mainly to static images with feedforward structures and lack spiking neuron mechanisms, leaving movie-stimulus representations in the mouse visual cortex poorly understood.
Approach
A feedforward spiking backbone (SEW-ResNet-style residual blocks with LIF neurons) augmented with a long-range feedback module (depthwise transposed convolution + fusion) in each recurrent stage, paired with a time-series RSA that correlates representational similarity vectors across sequential movie frames and a Spearman comparison to cortex.
Key results
- On Movie1, LoRaFB-SNet scores 0.520 vs CORnet 0.506, ResNet-2p-CPC 0.348 and SEW-ResNet 0.452, reaching 63.3% of the neural ceiling; on Movie2 it scores 0.283 (45.9% of ceiling).
- Versus SEW-ResNet trained on UCF101, LoRaFB-SNet improves TSRSA by +15.2% on Movie1 and +201.1% on Movie2.
- Under frame shuffling, LoRaFB-SNet's score drops by up to 46.5% while SEW-ResNet's drop stays under 9%, indicating sensitivity to dynamic information.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add long-range feedback connections to a UCF101-trained spiking network (ablation) | SEW-ResNet · Movie1 and Movie2 | TSRSA: +15.2% (Movie1); +201.1% (Movie2) | Helped |
| add spiking neurons to LoRaFB-SNet (ablation) | LoRaFB-SNet · Movie1 | TSRSA: 0.498 -> 0.520 | Helped |
| add long-range feedback connections (ablation) | SEW-ResNet · Movie1 | TSRSA: 0.452 -> 0.520 | Helped |
| pre-train LoRaFB-SNet on continuous rather than discontinuous UCF101 videos (ablation) | LoRaFB-SNet · Movie1 | TSRSA: 0.474 -> 0.524 | Helped |
| pre-train on UCF101 rather than ImageNet (ablation) | LoRaFB-SNet · Movie1 | TSRSA: 0.486 -> 0.520 | Helped |
| add long-range feedback connections to a spiking network trained on ImageNet | SEW-ResNet · Movie stimuli | TSRSA | No clear effect |
| replace original movie frames with Gaussian noise images | LoRaFB-SNet trained on UCF101 · Movie stimuli | TSRSA | Hurt |
| shuffle movie frame order | LoRaFB-SNet trained on UCF101 · Movie stimuli | TSRSA: 46.5% | Hurt |
| shuffle movie frame order | SEW-ResNet trained on UCF101 · Movie stimuli | TSRSA: lower than 9% | Hurt |
| use LoRaFB-SNet instead of CORnet | UCF101-trained models · Movie1 and Movie2 | TSRSA: +2.8% (Movie1); +26.8% (Movie2) | Helped |
| use LoRaFB-SNet instead of ResNet-2p-CPC | UCF101-trained models · Movie1 and Movie2 | TSRSA: +49.3% (Movie1); +64.4% (Movie2) | Helped |
Surprising findings
- SEW-ResNet trained on UCF101 performs worse than when trained on ImageNet, while LoRaFB-SNet improves with video pre-training.
- Training on discontinuous, randomly shuffled UCF101 frames yields lower scores than ImageNet training, suggesting static content of UCF101 is not closer to the test movies.
Limitations
- Evaluated on a single mouse neural dataset (Allen Brain Observatory subset) with only two movie stimuli plus one static scene set.
- Biological plausibility is limited to LIF neurons and a single feedback type; the paper acknowledges local recurrence and sophisticated neuron models are not included.
- Contributions of feedforward versus feedback connections to the encoding protocol remain unexplored, per the authors.
Experimental setup
Datasets: UCF101 ImageNet Allen Brain Observatory Visual Coding (Neuropixels subset)
Benchmarks: Allen Brain Observatory Visual Coding
Metrics: TSRSA score (Spearman correlation of Pearson similarity vect neural ceiling percentage R^2 from linear regression on individual neurons drop rate under frame shuffling
Baselines: CORnet ResNet-2p-CPC SEW-ResNet No-spike LoRaFB-SNet
Builds on: SEW-ResNet CORnet Representational Similarity Analysis SpikingJelly ResNet-2p-CPC
Closest papers by meaning
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.88)
- Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity (NeurIPS 2025 · similarity 0.87)
- Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributions (CVPR 2024 · similarity 0.86)
- Unveiling the Spatial-temporal Effective Receptive Fields of Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Sparse components distinguish visual pathways & their alignment to neural networks (ICLR 2025 · similarity 0.86)
- SIM: Surface-based fMRI Analysis for Inter-Subject Multimodal Decoding from Movie-Watching Experiments (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.