AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

The Loss Surface Of Deep Linear Networks Viewed Through The Algebraic Geometry Lens

Dhagash B. Mehta, Tianran Chen, Tingting Tang, Jonathan D. Hauenstein

TPAMI 2021 · Optimization › Implicit bias and generalization · approach: Shallow ReLU network convergence · 29 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add generalized L2 regularization to the deep linear loss (ablation)deep linear network loss · stationary pointsflat stationary pointsImproved
increase input dimension d_x from 2 to 5 (H=1, m=2)deep linear network, H=1 · isolated solutionsisolated complex solutions: 225 -> 225No clear effect
increase number of data points m from 2 to 20 (H=1)deep linear network, H=1 · isolated solutionsisolated complex solutions: 225 -> 225No clear effect
increase number of hidden layers from H=1 to H=2 (m=1)deep linear network · isolated solutionsisolated complex solutions: 33 -> 641Hurt
use BKK bound instead of classical Bezout bound (H=1, m=1, d_x=d_y=2)bound on solutions · upper bound on solution countbound size: 6561 -> 1024Improved
increase d_x from 2 to 10 (H=1, m=1)deep linear network · BKK boundBKK bound: 1024 -> 640000Hurt
increase d_y from 2 to 5 (H=1, m=1)deep linear network · complex solutionsisolated complex solutions: 33 -> 201Hurt
increase range of regularization Lambda beyond 1regularized deep linear network · real solutionsmean number of real solutionsHurt
increase number of data points mregularized deep linear network · real solutionsmean number of real solutionsImproved
use nonzero regularization with H=1, m=5, d_x=d_y=3regularized deep linear network · local minimalocal minima that are global: 7.16775 vs 7.13717Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy