Research map / Learning theory and algorithms / Prediction and decision losses
Critical Points of Linear Neural Networks: Analytical Forms and Landscape Properties
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a shallow linear network instead of a one-hidden-layer ReLU network | Square-loss neural-network landscape · Shallow networks | spurious local minima | Mixed |
| Capture larger singular values before smaller ones | Shallow linear network critical point · Shallow linear networks | square loss | Helped |
| Increase the output-layer rank to capture the next singular value | Optimal-order, non-global shallow linear critical point · Shallow linear networks | square loss | Helped |
| Replace a captured smaller singular direction with a skipped larger singular direction | Non-optimal-order shallow linear critical point · Shallow linear networks | square loss | Helped |
| Use ReLU activation and allow different active-data cones | One-hidden-layer network · ReLU example with X = diag(1,1), Y = diag(2,1) | square loss: \frac{1}{2} versus 2 | Mixed |
| Leave intermediate layers of certain deep non-global critical points unperturbed | Deep linear network · Remaining non-global-minimum critical points | descent direction | No clear effect |
Closest papers by meaning
- Pure and Spurious Critical Points: a Geometric Study of Linear Networks (ICLR 2020 · similarity 0.92)
- Embedding Principle of Loss Landscape of Deep Neural Networks (NeurIPS 2021 · similarity 0.90)
- Piecewise linear activations substantially shape the loss surfaces of neural networks (ICLR 2020 · similarity 0.90)
- Loss Landscape Characterization of Neural Networks without Over-Parametrization (NeurIPS 2024 · similarity 0.88)
- Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding (ICLR 2025 · similarity 0.88)
- SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.