Research map / Architectures and efficiency / RNNs and normalization
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase the fraction of trained input-output pathways | GDLN · CIFAR and MNIST transformed-task datasets | accuracy on untrained pathways: ≈40% | Helped |
| use small rather than large initialization (ablation) | GDLN pathway network · Routing task, untrained pathways | generalization | Helped |
| use large rather than small initialization (ablation) | GDLN pathway network · Routing task, untrained pathways | generalization | Hurt |
| increase pathway sharing (ablation) | GDLN routing network · Routing task | singular-value learning speed | Helped |
| train approximately 40% of input-output routes | GDLN routing network · Routing task, untrained routes | generalization: roughly 40% | Helped |
| use example-specific gating rather than a single always-active pathway (ablation) | GDLN · XOR | task solvability | Helped |
| use context-specific pathways rather than one always-active pathway (ablation) | GDLN · Nonlinear contextual classification | task solvability | Helped |
| use one always-active pathway without context-specific gating (ablation) | Deep linear network · Nonlinear contextual classification | task solvability | Hurt |
| use leaky-ReLU nonlinearity (ablation) | GDLN · MNIST transformed-task datasets, untrained pathways | accuracy: ≈25% | Helped |
Closest papers by meaning
- When Representations Align: Universality in Representation Learning Dynamics (ICML 2024 · similarity 0.89)
- How are linear representations learned? Exact solutions to the dynamics of abstraction (NeurIPS 2026 · similarity 0.89)
- Learning dynamics in linear recurrent neural networks (ICML 2025 · similarity 0.89)
- From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks (ICLR 2025 · similarity 0.89)
- How connectivity structure shapes rich and lazy learning in neural circuits (ICLR 2024 · similarity 0.89)
- A Dynamical Model of Neural Scaling Laws (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.