Research map / Learning theory and algorithms / Matrix and tensor methods
Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase network depth from shallow to deep | Multiclass classification on four classes · Four-class classification on the plane | presence of tripoints in class boundaries | Helped |
| Add or remove L2 regularization | Deep classifier · Four-class classification on the plane | class-boundary shape | No clear effect |
| Increase network depth | Autoencoder · MNIST and a one-dimensional dataset on the plane | denoising behavior | Helped |
| Use a very large depth | L2-regularized fully connected networks at global minima · Finite training data | recovery of true function rank | Hurt |
| Use intermediate depths as dataset size grows | L2-regularized fully connected networks · Data sampled from a piecewise-linear true function | true-rank recovery | Helped |
Closest papers by meaning
- Implicit bias of SGD in $L_2$-regularized linear DNNs: One-way jumps from high to low rank (ICLR 2024 · similarity 0.88)
- Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks (ICML 2025 · similarity 0.87)
- Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks (ICLR 2025 · similarity 0.87)
- Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss (ICML 2024 · similarity 0.87)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.87)
- DeNetDM: Debiasing by Network Depth Modulation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.