Research map / Optimization / Implicit bias and generalization
Training Linear Neural Networks: Non-Local Convergence and Complexity Results
Armin Eftekhari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Restrict gradient flow initialization to the stable set N_{N,alpha} (balanced init, end-to-end product positively correlated with target and bounded away from zero) | gradient flow for deep linear networks with one width-1 layer · Theoretical (whitened data, r=1, Assumption 3.6) | convergence to global minimizer | Improved |
| Increase network depth N in linear network trained by gradient flow | faraway (non-local) convergence rate of gradient flow · Theoretical bound (Theorem 4.4) and numerical example (Figure 1, dx=5, dy=1, m=50) | distance to target / convergence rate | Improved |
| Increase network depth N in linear network trained by gradient flow | nearby (local) convergence rate of gradient flow · Theoretical bound (Theorem 4.4) | convergence rate | No clear effect |
| Use a balanced (rather than unbalanced) initialization for gradient flow | gradient flow on linear networks · Theoretical (Section 3) | guarantee of avoiding rank-degenerate limit points | Improved |
Closest papers by meaning
- A Non-Local Convergence Analysis of Gradient Flow for Deep Linear Networks (TPAMI 2026 · similarity 0.92)
- On Non-local Convergence Analysis of Deep Linear Networks (ICML 2022 · similarity 0.90)
- A unifying view on implicit bias in training linear neural networks (ICLR 2021 · similarity 0.89)
- The Loss Landscape of Deep Linear Neural Networks: a Second-order Analysis (JMLR 2024 · similarity 0.89)
- Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks (NeurIPS 2025 · similarity 0.87)
- Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
