Research map / Architectures and efficiency / RNNs and normalization
On the Power of Over-parametrization in Neural Networks with Quadratic Activation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use width k ≥ d | Quadratic-activation shallow network · Arbitrary training dataset | loss landscape | Helped |
| Use width k with k(k+1)/2 > n and add an arbitrarily small random PSD perturbation | Quadratic-activation shallow network with k < d · Arbitrary training dataset | loss landscape | Helped |
| Apply weight decay to bound the weight Frobenius norm | Quadratic-activation shallow network · Gaussian inputs | generalization gap | Helped |
Closest papers by meaning
- Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions (NeurIPS 2020 · similarity 0.88)
- How degenerate is the parametrization of neural networks with the ReLU activation function? (NeurIPS 2019 · similarity 0.87)
- The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks (NeurIPS 2025 · similarity 0.86)
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.84)
- Magnitude Invariant Parametrizations Improve Hypernetwork Learning (ICLR 2024 · similarity 0.84)
- Quadratic models for understanding catapult dynamics of neural networks (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.