Research map / Architectures and efficiency / RNNs and normalization
Recurrent neural network dynamical systems for biological vision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CordsNet, a hybrid architecture that replaces CNN convolutional layers with continuous-time convolutional RNN dynamics, trained via a cheaper CNN-based initialization, and analyzes its dynamics, robustness, and fit to V4/IT neural activity.
Problem
CNNs dominate vision neuroscience but lack the continuous-time dynamics of biological recurrent circuits, while dynamical RNNs cannot efficiently process images at scale.
Approach
Replace every convolutional layer with a continuous-time recurrent layer whose input and recurrent weights are convolutions, and initialize it by folding batch norm from a feedforward CNN, annealing a linear RNN to a PReLU nonlinearity, then fine-tuning end-to-end.
Key results
- Fine-tuned CordsNet-R8 reaches 63.16% ImageNet validation accuracy, versus 23.21% when trained directly with matched time (control C) and 0.26% for fully-connected RNN control.
- CordsNet-R8 activation deviations under noise are attenuated by more than an order of magnitude relative to the feedforward CNN, and it maintains accuracy at high noise levels.
- All CordsNet models score higher on unshuffled than temporally-shuffled neural data across V4 and IT (paired t-test, all p < 10^-4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the proposed initialization method instead of direct CordsNet training (ablation) | CordsNet · MNIST | accuracy (%): 99.40 -> 99.37 | Hurt |
| add the proposed initialization method instead of direct CordsNet training (ablation) | CordsNet · F-MNIST | accuracy (%): 91.82 -> 94.26 | Helped |
| add the proposed initialization method instead of direct CordsNet training (ablation) | CordsNet · CIFAR-10 | accuracy (%): 83.15 -> 91.57 | Helped |
| add the proposed initialization method instead of direct CordsNet training (ablation) | CordsNet · CIFAR-100 | accuracy (%): 51.34 -> 66.89 | Helped |
| add the proposed initialization method instead of direct CordsNet training (ablation) | CordsNet · ImageNet validation | accuracy (%): 23.21 -> 41.24 | Helped |
| fine-tune after initialization (ablation) | CordsNet-R8 · MNIST | accuracy (%): 99.37 -> 99.36 | No clear effect |
| fine-tune after initialization (ablation) | CordsNet-R8 · F-MNIST | accuracy (%): 94.26 -> 95.88 | Helped |
| fine-tune after initialization (ablation) | CordsNet-R8 · CIFAR-10 | accuracy (%): 91.57 -> 94.56 | Helped |
| fine-tune after initialization (ablation) | CordsNet-R8 · CIFAR-100 | accuracy (%): 66.89 -> 77.32 | Helped |
| fine-tune after initialization (ablation) | CordsNet-R8 · ImageNet validation | accuracy (%): 41.24 -> 57.90 | Helped |
| use the continuous-time recurrent architecture rather than a feedforward CNN | CordsNet-R8 · MNIST | accuracy (%): 99.74 -> 99.36 | Hurt |
| use the continuous-time recurrent architecture rather than a feedforward CNN | CordsNet-R8 · F-MNIST | accuracy (%): 96.13 -> 95.88 | Hurt |
Surprising findings
- Fully-connected RNNs perform remarkably poorly on ImageNet, attaining near-zero accuracy, due to poor scaling with image size.
- Fine-tuned CordsNets remain slightly below their feedforward CNN counterparts in accuracy.
Limitations
- Substantial memory requirements and high MAC counts from simulating 300 time steps
- Difficult dynamical analysis because recurrent kernel cannot be expanded into full matrix
- Fine-tuned accuracy still below feedforward CNN counterparts
Experimental setup
Datasets: ImageNet MNIST Fashion-MNIST CIFAR-10 CIFAR-100 neural recordings from V4 and IT (Brain-Score)
Benchmarks: ImageNet MNIST Fashion-MNIST CIFAR-10 CIFAR-100 Brain-Score
Metrics: test accuracy validation accuracy Pearson correlation mean-squared deviation MACs parameter count
Baselines: CORNet-RT fully-connected RNN low-rank RNN sparse RNN feedforward CNN ResNet-18 ViT-H-14
Builds on: ResNet CORnet-RT Brain-Score low-rank RNN Arnoldi iteration
Closest papers by meaning
- Inference of Neural Dynamics Using Switching Recurrent Neural Networks (NeurIPS 2024 · similarity 0.89)
- Learning Dynamics of RNNs in Closed-Loop Environments (NeurIPS 2025 · similarity 0.89)
- RTify: Aligning Deep Neural Networks with Human Behavioral Decisions (NeurIPS 2024 · similarity 0.89)
- Mechanistic Interpretability of RNNs emulating Hidden Markov Models (NeurIPS 2025 · similarity 0.88)
- Unconditional stability of a recurrent neural circuit implementing divisive normalization (NeurIPS 2024 · similarity 0.88)
- Learning dynamics in linear recurrent neural networks (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.