Research map / Optimization / Implicit bias and generalization
The asymptotic spectrum of the Hessian of DNN throughout training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the infinite-width fixed-NTK regime | DNN Hessian decomposition H = I + S · Sequential infinite-width limit during training | Hessian moments | Helped |
| Characterize I through the NTK Gram matrix | DNN Hessian · Finite training dataset, fixed-NTK limit | I moment spectrum | Helped |
| Separate I and S using their asymptotic mutual orthogonality | DNN Hessian · Sequential infinite-width limit during training | Hessian moments | Helped |
| Include the residual Hessian term S | DNN Hessian · Sequential infinite-width limit during training | Higher moments of S | No clear effect |
| Train toward convergence so the loss gradient shrinks | Residual Hessian term S · During training | Frobenius norm of S | Helped |
| Normalize the nonlinearity | DNN loss surface · 14x14 MNIST and data uniform on the unit disk | Narrow valley structure | Helped |
Closest papers by meaning
- On the linearity of large non-linear models: when and why the tangent kernel is constant (NeurIPS 2020 · similarity 0.91)
- Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study (NeurIPS 2022 · similarity 0.90)
- Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability (NeurIPS 2025 · similarity 0.89)
- On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory (NeurIPS 2024 · similarity 0.88)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.88)
- A generalized neural tangent kernel for surrogate gradient learning (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.