Research map / Learning theory and algorithms / Kernels and regression theory
Spectral Asymptotics of Neural Network Jacobians: Convergency, Universality, and Phase Transition
Huiqin Li, Guangming Pan, Yanqing Yin
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives the limiting spectral distribution and second-order spectral fluctuations of neural network Jacobians, showing universality in the weight distribution and a phase transition driven by the activation function.
Problem
Spectral behavior of neural network Jacobian matrices is not well characterized; the paper asks what the limiting spectral distribution and fluctuations of the Jacobian's spectrum are.
Approach
Uses random matrix theory tools to establish the limiting spectral distribution and second-order (fluctuation) behavior of the Jacobian, showing universality with respect to the weight distribution and a phase transition in spectral behavior determined by the activation function.
Key results
- Establishes the limiting spectral distribution of neural network Jacobian matrices (per abstract).
- Derives second-order fluctuations of the Jacobian spectrum (per abstract).
- Reports universality with respect to the weight distribution and a phase transition driven by the activation function (per abstract).
Closest papers by meaning
- Free Probability for predicting the performance of feed-forward fully connected neural networks (NeurIPS 2022 · similarity 0.88)
- An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network (ICML 2024 · similarity 0.85)
- Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos (ICLR 2025 · similarity 0.85)
- Random Matrix Theory of Early-Stopped Gradient Flow: A Transient BBP Scenario (NeurIPS 2026 · similarity 0.85)
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra (ICLR 2025 · similarity 0.84)
- Models of Heavy-Tailed Mechanistic Universality (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
