Research map / Architectures and efficiency / RNNs and normalization
The interplay between randomness and structure during learning in RNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add nonzero random initial connectivity (g>0) vs g=0 | nonlinear RNN · Flip-flop, Mante, Romo | convergence time | Helped |
| very large initial connectivity g (chaotic regime) | nonlinear RNN · Flip-flop, Mante, Romo | convergence time | Hurt |
| keep correlation between W0 and Delta W (vs shuffling W0) (ablation) | trained RNN · Flip-flop, Mante, Romo | loss | Helped |
| keep W0 and truncate only Delta W (vs truncating whole W) (ablation) | 2-layer LSTM · Sentiment analysis | accuracy at low rank: rank 10 vs more than half of network rank | Helped |
| increase g in linear RNN (theory) | linear RNN simple task · Simplified input-output task | convergence time | Helped |
Closest papers by meaning
- Learning Reveals Invisible Structure in Low-Rank RNNs (NeurIPS 2026 · similarity 0.92)
- How connectivity structure shapes rich and lazy learning in neural circuits (ICLR 2024 · similarity 0.91)
- Learning dynamics in linear recurrent neural networks (ICML 2025 · similarity 0.88)
- Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks (NeurIPS 2025 · similarity 0.88)
- Dynamical phases of short-term memory mechanisms in RNNs (ICML 2025 · similarity 0.88)
- Inferring stochastic low-rank recurrent neural networks from neural data (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.