Research map / Learning theory and algorithms / Prediction and decision losses
Adding One Neuron Can Eliminate All Bad Local Minima
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a skip-connected exponential neuron with a quadratic regularizer | Any neural network for binary classification · Realizable binary classification dataset | local-minimum optimality | Helped |
| Add one exponential neuron per layer with the stated regularizer | Fully connected feedforward neural network · Realizable binary classification dataset | local-minimum optimality | Helped |
| Replace the exponential neuron with a degree-p monomial neuron | Any neural network for binary classification · Dataset classifiable by a polynomial of degree t, with p ≥ t | local-minimum optimality | Helped |
| Use the augmented exponential-neuron objective with a convex loss | Any neural network for binary classification · Dataset that may contain conflicting labels for the same feature vector | local-minimum optimality | Helped |
| Add the exponential neuron and its associated regularizer | Original neural network · Realizable binary classification dataset | training misclassification error | Helped |
Closest papers by meaning
- Understanding the Loss Surface of Neural Networks for Binary Classification (ICML 2018 · similarity 0.89)
- Small nonlinearities in activation functions create bad local minima in neural networks (ICLR 2019 · similarity 0.88)
- Piecewise linear activations substantially shape the loss surfaces of neural networks (ICLR 2020 · similarity 0.87)
- Spurious Local Minima are Common in Two-Layer ReLU Neural Networks (ICML 2018 · similarity 0.87)
- The Loss Surface of Deep and Wide Neural Networks (ICML 2017 · similarity 0.87)
- Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.