Research map / Optimization / Implicit bias and generalization
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use small random initialization instead of large initialization (ablation) | Gradient descent for low-rank matrix reconstruction · Low-rank matrix reconstruction, r = 180 | test error: 10^{-3} vs. 0.5 | Helped |
| Increase factorization width from 5 to 10 (ablation) | Gradient descent with alpha=10^{-3} · Low-rank matrix reconstruction, r from 5 to 30 | iterations until test error falls below 10^{-4}: 5 to 10 | Helped |
| Increase factorization width beyond 10 (ablation) | Gradient descent with alpha=10^{-3} · Low-rank matrix reconstruction, r from 5 to 30 | iterations until test error falls below 10^{-4}: 10 to 30 | No clear effect |
| Increase factorization width r (ablation) | Gradient descent with small initialization · Low-rank matrix reconstruction | iterations to align with the leading spectral subspace | Helped |
| Decrease initialization scale alpha (ablation) | Gradient descent for low-rank matrix reconstruction · Low-rank matrix reconstruction, r = 180 | test error: \alpha^{21/16} | Helped |
| Use large initialization in the lazy-training regime (ablation) | Gradient descent for low-rank matrix reconstruction · Low-rank matrix reconstruction, r = 180 | test error: 0.5 | No clear effect |
| Use small random initialization and vanilla gradient descent under the stated RIP and step-size assumptions | Low-rank matrix reconstruction · Low-rank matrix reconstruction, any r covered by the theorems | reconstruction error: \alpha^{21/16} | Helped |
| Overparameterize the factorization | Small-initialization gradient descent · Low-rank matrix reconstruction | time to spectral alignment | Helped |
Closest papers by meaning
- How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization (ICLR 2024 · similarity 0.90)
- Free Decompression with Algebraic Spectral Curves (NeurIPS 2026 · similarity 0.88)
- On the Crucial Role of Initialization for Matrix Factorization (ICLR 2025 · similarity 0.88)
- Random Matrix Theory of Early-Stopped Gradient Flow: A Transient BBP Scenario (NeurIPS 2026 · similarity 0.88)
- Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions (ICML 2024 · similarity 0.87)
- Globally Q-linear Gauss-Newton Method for Overparameterized Non-convex Matrix Sensing (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.