Research map / Learning theory and algorithms / Prediction and decision losses
Understanding the Loss Surface of Neural Networks for Binary Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use smooth hinge loss, strictly convex analytic neurons, and a shortcut-like architecture under the stated data assumptions | Binary classification networks · Linear-subspace data | training error at every local minimum: training error is zero | Helped |
| Use smooth hinge loss, increasing neurons, and a shortcut-like architecture | Binary classification networks · Linearly separable data | training error at every local minimum: training error is zero | Helped |
| Use quadratic activation with M > r | Shortcut-like binary classifier on linear-subspace data · Linear-subspace data | training error at every local minimum: training error is zero | Helped |
| Use ReLU-class neurons in the single-layer component (ablation) | Smooth-hinge shortcut-like classifier · Binary classification | training error at a local minimum: \hat{R}_{n}(\bm{\theta}^{*})\ge\frac{\min\{n_{+},n_{-}\}}{n} | Hurt |
| Use Leaky-ReLU-class neurons in the single-layer component (ablation) | Smooth-hinge shortcut-like classifier · Binary classification | training error at a local minimum: non-zero training error | Hurt |
| Use sigmoid-class neurons in the single-layer component (ablation) | Smooth-hinge shortcut-like classifier · Binary classification | training error at a local minimum: \hat{R}_{2n}(\bm{\theta}^{*})\ge\frac{\min\{n_{-},n_{+}\}}{2n} | Hurt |
| Use quadratic-family neurons instead of neurons with positive derivatives (ablation) | Shortcut-like classifier · Linearly separable data | training error at a local minimum: \hat{R}_{n}(\bm{\theta}^{*})\ge\frac{\min\{n_{+},n_{-}\}}{n} | Hurt |
| Remove the shortcut-like connection (ablation) | Smooth-hinge feedforward ReLU classifier · Binary classification | training error at a local minimum: \hat{R}_{n}(\bm{\theta}^{*})\ge\frac{\min\{n_{+},n_{-}\}}{n} | Hurt |
| Replace the shortcut-like connection with an identity shortcut (ablation) | Smooth-hinge network with ReLU deep component · Binary classification | training error at a local minimum: non-zero training error | Hurt |
| Replace smooth hinge loss with quadratic loss (ablation) | Linear classifier · Linearly separable distribution | population misclassification error at a global minimum: \ge 1/16 | Hurt |
| Replace smooth hinge loss with logistic loss (ablation) | Shortcut-like network under linear-subspace assumptions · Linear-subspace data | existence of local minima: every critical point is a saddle point; there are no local minima | Hurt |
| Use logistic loss in a feedforward network (ablation) | Feedforward binary classifier · Datasets containing both positive and negative samples | training error at every critical point: \hat{R}_{n}(\bm{\theta}^{*})>0 | Hurt |
Closest papers by meaning
- Adding One Neuron Can Eliminate All Bad Local Minima (NeurIPS 2018 · similarity 0.89)
- A Universal Growth Rate for Learning with Smooth Surrogate Losses (NeurIPS 2024 · similarity 0.88)
- An Empirical study of Binary Neural Networks' Optimisation (ICLR 2019 · similarity 0.88)
- Are All Losses Created Equal: A Neural Collapse Perspective (NeurIPS 2022 · similarity 0.88)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.87)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.