Research map / Architectures and efficiency / RNNs and normalization
Invariance of Weight Distributions in Rectified MLPs
Russell Tsuchida, Fred Roosta, Marcus Gallagher
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Closest papers by meaning
- Deep Equals Shallow for ReLU Networks in Kernel Regimes (ICLR 2021 · similarity 0.88)
- Initialization of ReLUs for Dynamical Isometry (NeurIPS 2019 · similarity 0.87)
- Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes (ICLR 2024 · similarity 0.87)
- Sinusoidal Initialization, Time for a New Start (NeurIPS 2025 · similarity 0.86)
- Deep Kernel Posterior Learning under Infinite Variance Prior Weights (ICLR 2025 · similarity 0.86)
- Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
