Research map / Optimization / Implicit bias and generalization
Gradient Descent Finds Global Minima of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use residual connections with residual scaling c_res/(H√m) | Deep neural network training with gradient descent · Training data (nondegenerate inputs) | required width dependence on depth: 2^{O(H)} -> poly(H) | Helped |
| Use a smooth activation | Fully-connected network at depth H=1 · Training data | required width dependence on sample size: n^6 -> n^4 | Helped |
| Extend the residual architecture with convolutional layers | Gradient descent on convolutional ResNet · Training data (nondegenerate inputs) | training loss | Helped |
Closest papers by meaning
- A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks (ICLR 2019 · similarity 0.91)
- Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs (ICML 2017 · similarity 0.91)
- A Convergence Theory for Deep Learning via Over-Parameterization (ICML 2019 · similarity 0.91)
- Global Optimality in Neural Network Training (CVPR 2017 · similarity 0.90)
- Block Coordinate Descent for Neural Networks Provably Finds Global Minima (NeurIPS 2025 · similarity 0.89)
- Generalization of Scaled Deep ResNets in the Mean-Field Regime (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.