Research map / Learning theory and algorithms / Matrix and tensor methods
Non-Gaussian Tensor Programs
Eugene Golikov, Greg Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| sample input/output (vector-like) weights from Rademacher instead of Gaussian with equal variance | 1-hidden-layer tied-weight network f(xi)=U^T phi(U xi)/n · width to infinity, phi(x)=x*1[|x|<=1/2] | network output f(1) | Hurt |
| sample vector-like weights from +-1 instead of standard Gaussian | NNGP of f=V phi(U xi)/sqrt(n) with indicator phi · infinite-width GP limit | limiting Gaussian process kernel | Hurt |
| swap Gaussian for non-Gaussian matrix-like (hidden) weights with the same mean and variance | Tensor Programs / wide neural networks · infinite-width limit | limiting training behavior | No clear effect |
| swap Gaussian for non-Gaussian initial vectors in a program | Tensor Programs · infinite-width limit | program limit | Mixed |
| use a different initialization distribution (mean 0, variance 1) for abc-parametrization weights | abc-parametrization of MLPs · feature learning/kernel/stable/nontrivial regimes | regime classification | No clear effect |
| add bias parameters to the MLP | Principle 1 universality claim for biasless MLP · wide MLP initialization | validity of universality principle | Hurt |
Closest papers by meaning
- Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs (NeurIPS 2025 · similarity 0.85)
- How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers (ICML 2024 · similarity 0.85)
- On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory (NeurIPS 2024 · similarity 0.84)
- Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structure (ICLR 2025 · similarity 0.84)
- Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.84)
- Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
