Research map / Optimization / Implicit bias and generalization
Gradient Descent Learns One-hidden-layer CNN: Don’t be Afraid of Spurious Local Minima
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the teacher output-weight ratio (sum squared divided by squared norm) | Random-init gradient descent with weight normalization · Synthetic Gaussian, fixed k | global-minimum convergence probability: For k=25: 0.50 -> 1 | Helped |
| Increase the number of output weights k | Random-init gradient descent with weight normalization · Synthetic Gaussian, fixed ratio | global-minimum convergence probability: At ratio 25: 1 -> 0.90 | Hurt |
| Use the sign combination giving positive alignment and angle below pi/2 | Gradient descent for the one-hidden-layer CNN · Gaussian population loss | convergence to global minimum | Helped |
| Initialize with negative alignment in the stated small-ratio regime | Gradient descent for the one-hidden-layer CNN · Gaussian population loss | convergence to spurious local minimum | Hurt |
| Use gradient descent with weight normalization from the specified random initialization | One-hidden-layer CNN with both layers learned · Gaussian population loss | parameter recovery | Helped |
| Run multiple sign trials/restarts | Random-init gradient descent · Gaussian population loss | probability of recovering true parameters | Helped |
| Proceed from the initial low-signal phase into the strong-signal phase | Gradient descent with weight normalization · Gaussian population loss | convergence rate | Helped |
Closest papers by meaning
- Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs (ICML 2017 · similarity 0.90)
- A Convergence Theory for Deep Learning via Over-Parameterization (ICML 2019 · similarity 0.89)
- Spurious Local Minima are Common in Two-Layer ReLU Neural Networks (ICML 2018 · similarity 0.89)
- Small nonlinearities in activation functions create bad local minima in neural networks (ICLR 2019 · similarity 0.89)
- How does Gradient Descent Learn Features --- A Local Analysis for Regularized Two-Layer Neural Networks (NeurIPS 2024 · similarity 0.87)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.