Research map / Optimization / Implicit bias and generalization
Convergence of the Gradient Flow for Shallow ReLU Networks on Weakly Interacting Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that one-hidden-layer ReLU networks trained by gradient flow converge globally with high probability when width is on the order of log(n), exploiting low correlation of high-dimensional inputs, via a Polyak–Łojasiewicz analysis.
Problem
Global convergence of gradient flow for one-hidden-layer ReLU networks trained on n data points, and how much width is needed for it.
Approach
Uses the high dimensionality of the ambient space, which makes input samples weakly correlated, to show that width of order log(n) suffices for global convergence under a Polyak–Łojasiewicz argument along the gradient flow trajectory.
Key results
- Width of order log(n) neurons suffices for global convergence of gradient flow with high probability for one-hidden-layer ReLU networks on n data points.
Closest papers by meaning
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.89)
- Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding (ICLR 2025 · similarity 0.89)
- Transformative or Conservative? Conservation laws for ResNets and Transformers (ICML 2025 · similarity 0.89)
- Topological obstruction to the training of shallow ReLU neural networks (NeurIPS 2024 · similarity 0.89)
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.89)
- SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.