Research map / Architectures and efficiency / RNNs and normalization
Cortical microcircuits as gated-recurrent neural networks
Rui Costa, Ioannis Alexandros Assael, Brendan Shillingford, Nando de Freitas, TIm Vogels
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace multiplicative LSTM gating with subtractive inhibitory gating | subLSTM · Penn Treebank test | perplexity: 222.80 vs 215.93; 91.46 vs 88.39; 79.59 vs 74.60; 76.17 vs 64.34 | Mixed |
| replace multiplicative LSTM gating with subtractive inhibitory gating | subLSTM · Wikitext-2 test | perplexity: 268.33 vs 271.44; 103.36 vs 102.77; 89.00 vs 86.15; 78.92 vs 74.27 | Mixed |
| use a fixed learned memory decay instead of a learned forget gate (ablation) | subLSTM · Penn Treebank test | perplexity: 213.86 vs 222.80; 91.84 vs 91.46; 81.97 vs 79.59; 70.58 vs 76.17 | Mixed |
| use a fixed learned memory decay instead of a learned forget gate (ablation) | subLSTM · Wikitext-2 test | perplexity: 259.89 vs 268.33; 105.06 vs 103.36; 94.33 vs 89.00; 79.49 vs 78.92 | Mixed |
| use subtractive rather than multiplicative gating | subLSTM · sequential MNIST test | classification accuracy | No clear effect |
Closest papers by meaning
- Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons (NeurIPS 2026 · similarity 0.88)
- Unconditional stability of a recurrent neural circuit implementing divisive normalization (NeurIPS 2024 · similarity 0.87)
- The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks. (ICLR 2024 · similarity 0.86)
- Dynamical phases of short-term memory mechanisms in RNNs (ICML 2025 · similarity 0.85)
- Recurrent neural network dynamical systems for biological vision (NeurIPS 2024 · similarity 0.85)
- Traveling Waves Encode The Recent Past and Enhance Sequence Learning (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
