Research map / Learning theory and algorithms / Prediction and decision losses
Deep Linear Networks with Arbitrary Loss: All Local Minima Are Global
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a convex differentiable loss and make every hidden layer at least as wide as the smaller of the input and output layers | Deep linear networks · Arbitrary data and targets | local minima optimality | Helped |
| Replace a differentiable loss with a convex, Lipschitz, nondifferentiable loss | Deep linear networks · Two-layer example | local minima optimality | Hurt |
| Make a hidden layer narrower than the smaller of the input and output layers | Deep linear networks with differentiable objectives · General matrix-product objective | first-order optimality of the product | Hurt |
| Remove convexity of the differentiable objective | Deep linear networks · Example f(x,y) = x² − y² | local minima optimality | Hurt |
Closest papers by meaning
- The Loss Surface of Deep and Wide Neural Networks (ICML 2017 · similarity 0.92)
- Small nonlinearities in activation functions create bad local minima in neural networks (ICLR 2019 · similarity 0.90)
- Global Optimality in Neural Network Training (CVPR 2017 · similarity 0.89)
- Unique Properties of Flat Minima in Deep Networks (ICML 2020 · similarity 0.89)
- Implicit bias of SGD in $L_2$-regularized linear DNNs: One-way jumps from high to low rank (ICLR 2024 · similarity 0.88)
- Block Coordinate Descent for Neural Networks Provably Finds Global Minima (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.