Research map / Learning theory and algorithms / Kernels and regression theory
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Randomize two-layer network weights by multiplying each neuron's weight vector by a random sign Σrr ∈ {±1} (sign randomization), so the quadratic term is unaffected while the linear NTK term has mean zero | two-layer over-parametrized neural network · theoretical analysis (landscape / coupling) | dominance of quadratic term over linear NTK term | Helped |
| Use ℓ2,4 norm regularization on the first-layer weights in the randomized risk | randomized two-layer network risk · theoretical analysis (landscape) | localization of second-order stationary points and bounded loss | Helped |
| Use a large learning rate (up to O(m^{1/4})) in the randomized network training, staying within the O(1) ℓ2,4 ball where coupling holds | randomized two-layer network training · theoretical analysis | allowed learning rate | Helped |
| Replace Frobenius-norm generalization with operator-norm based bound for quadratic model feature maps | quadratic model generalization · theoretical bound for learning polynomials on isotropic data | sample complexity relative to linear NTK: n_Q/n_L = eO(p/d) | Helped |
Closest papers by meaning
- What can linearized neural networks actually say about generalization? (NeurIPS 2021 · similarity 0.90)
- On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models (ICML 2021 · similarity 0.90)
- Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks (ICML 2025 · similarity 0.88)
- How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers (ICML 2024 · similarity 0.88)
- On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory (NeurIPS 2024 · similarity 0.88)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.