Research map / Learning theory and algorithms / Kernels and regression theory
Analytic Insights into Structure and Rank of Neural Network Hessian Maps
Sidak Pal Singh, Gregor Bachmann, Thomas Hofmann
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the linear-network rank formula to choose the Hessian approximation rank | ReLU network Hessian · ReLU network (setting unspecified) | relative reconstruction error: 0.002% error | Improved |
| Increase network width and depth | Neural network architectures · Simulated architectures | rank/number of parameters | Mixed |
| Enable bias | Deep linear network · Simulated architectures across depth | rank/number of parameters | Hurt |
| Train toward convergence so the residual covariance approaches zero | Deep neural network · Training | functional Hessian rank | Hurt |
Closest papers by meaning
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.92)
- Non-Singularity of the Gradient Descent Map for Neural Networks with Piecewise Analytic Activations (NeurIPS 2025 · similarity 0.87)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.86)
- Loss Landscape Characterization of Neural Networks without Over-Parametrization (NeurIPS 2024 · similarity 0.86)
- Simplicity Bias via Global Convergence of Sharpness Minimization (ICML 2024 · similarity 0.86)
- Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
