Research map / Optimization / Implicit bias and generalization
Implicit Sparse Regularization: The Impact of Depth and Early Stopping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use early stopping instead of running gradient descent without a stopping window | Depth-N diagonal linear network for sparse regression · Noisy sparse regression with correlated designs | sparse recovery | Helped |
| Increase depth N | Early-stopped depth-N diagonal linear network · Sparse regression | early-stopping window size | Helped |
| Increase depth N while holding initialization w₀ and the step-size schedule comparable (ablation) | Depth-N diagonal linear network · Sparse recovery simulation | recovery | Helped |
| Use initialization α^N = 2×10^{-3} rather than a sufficiently small initialization (ablation) | Depth-2 diagonal linear network · Sparse recovery simulation | recovery | Hurt |
| Use a larger depth with the same relatively large initialization (ablation) | Depth-N diagonal linear network · Sparse recovery simulation | recovery | Helped |
| Increase depth N (ablation) | Depth-N diagonal linear network · Sparse recovery simulation | iterations to recover | Hurt |
| Increase depth N (ablation) | Depth-N diagonal linear network · Sparse recovery with targets having different coordinate magnitudes | separation of learning phases | Helped |
| Use large initialization and deploy early stopping (ablation) | Depth-N diagonal linear network · Ridge-regression setting | connection to ridge regression | Helped |
Closest papers by meaning
- Implicit Regularization for Optimal Sparse Recovery (NeurIPS 2019 · similarity 0.92)
- Implicit Regularization in Deep Matrix Factorization (NeurIPS 2019 · similarity 0.91)
- Implicit Regularization of Decentralized Gradient Descent for Sparse Regression (NeurIPS 2024 · similarity 0.90)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.90)
- Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent (ICML 2025 · similarity 0.89)
- Estimating Implicit Regularization in Deep Learning (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.