Research map / Learning theory and algorithms / Matrix and tensor methods
Geometry of Neural Network Loss Surfaces via Random Matrix Theory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase energy epsilon of critical point (more Wigner-like H1 noise) in Wishart+Wigner Hessian approximation | Hessian spectrum (H0+H1) · Analytic spectral density, phi=1/4, 1/2, 3/4 | spectrum shape / negative eigenvalues | Mixed |
| Increase phi (ratio of parameters to data points) | Energy of minimizers in memorization task · tanh single-hidden-layer, n=16, random data | epsilon_c (energy of minimizers) | Hurt |
| Use tanh nonlinearity instead of linear network | Single-hidden-layer network energy of minimizers · memorization task, phi between 0 and 1 | epsilon_c functional form | Helped |
| Model index scaling for small index: alpha ~ |eps-eps_c|^(3/2) | Critical point index vs energy · tanh memorization experiments (~50k runs) | index scaling exponent | No clear effect |
| Replace Wishart+Wigner by refined H1 product-Wishart plus sigma-restored MP H0 approximation | Hessian spectral approximation · Analytic, single-hidden-layer ReLU | index scaling exponent | No clear effect |
Closest papers by meaning
- A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization (AAAI 2021 · similarity 0.89)
- Beyond Spectra: Eigenvector Overlaps in Loss Geometry (ICLR 2026 · similarity 0.89)
- The Spectrum of the Fisher Information Matrix of a Single-Hidden-Layer Neural Network (NeurIPS 2018 · similarity 0.89)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.86)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.86)
- Simplicity Bias via Global Convergence of Sharpness Minimization (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.