Research map / Architectures and efficiency / RNNs and normalization
Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an overparameterized student with m ≥ d | Quadratic-activation teacher-student learning · Phase retrieval (m^*=1), Gaussian samples | sample complexity threshold (α_c): α_c=2; n > 2d | Helped |
| Increase teacher width m^* while keeping m^*<d | Overparameterized quadratic student · Gaussian teacher-student learning | sample complexity threshold (α_c): α_c = m^*+1 | Hurt |
| Train on the population loss when m^*<d | Quadratic student with m ≥ d · Population loss | asymptotic loss decay: E(A(t)) ∼ 1/(16t^2) | Mixed |
| Train on the population loss when m^*≥d | Quadratic student with m ≥ d · Population loss, orthonormal teacher vectors | asymptotic loss decay: E[A(t)] ∼ 1/(2d)e^{-2(2d+4)t} | Helped |
Closest papers by meaning
- On the Power of Over-parametrization in Neural Networks with Quadratic Activation (ICML 2018 · similarity 0.88)
- Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws (NeurIPS 2025 · similarity 0.88)
- The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks (NeurIPS 2025 · similarity 0.87)
- Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks (NeurIPS 2024 · similarity 0.86)
- Quadratic models for understanding catapult dynamics of neural networks (ICLR 2024 · similarity 0.86)
- The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.