Research map / Architectures and efficiency / RNNs and normalization
Optimal Architectures in a Solvable Model of Deep Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase sparsity (reduce f) | deep pseudo-inverse network · infinitely deep homogeneous network, overlap dynamics | basin of attraction of noiseless fixed point | Helped |
| add depth (finite deep network vs single infinitely wide layer) | pseudo-inverse processing network · readout error with alpha=0.2 vs single infinite layer (m0=0.6) | readout error | Helped |
| increase number of processing layers at fixed total neuron count | pseudo-inverse network with Nt=N0 · mean readout error vs depth L | mean error E | Mixed |
| increase total neuron count (expansion factor) | pseudo-inverse network · optimal depth and error vs expansion factor | minimal mean error E_opt and optimal depth | Helped |
| increase input load P (number of stored templates) | pseudo-inverse network · optimal layout vs load | optimal depth L_opt | Hurt |
| include induced-correlation term Q_l in field variance (ablation) | mean field recursion for PI network · mean-field theory accuracy vs prior analysis (dense f=0.5) | agreement with simulations | Helped |
| deep vs single layer on MNIST autoencoder reconstruction | pseudo-inverse autoencoder (10 layers vs single layer, same neurons) · MNIST 3-digit templates, qualitative reconstruction | reconstruction of noiseless template | Helped |
Closest papers by meaning
- Is Deeper Better only when Shallow is Good? (NeurIPS 2019 · similarity 0.87)
- The Neural Race Reduction: Dynamics of Abstraction in Gated Networks (ICML 2022 · similarity 0.87)
- The Loss Surface of Deep and Wide Neural Networks (ICML 2017 · similarity 0.86)
- Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons (NeurIPS 2026 · similarity 0.86)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.85)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.