Research map / Optimization / Implicit bias and generalization
The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the step size | Single-hidden-layer multivariate ReLU network trained with GD · Synthetic regression | stability norm | Helped |
| Increase the step size | Single-hidden-layer multivariate ReLU network trained with SGD · MNIST 0-vs-1 validation set | classification accuracy | Helped |
| Increase the step size | Single-hidden-layer multivariate ReLU network trained with GD · Synthetic regression | predictor smoothness | Helped |
| Increase the step size | Single-hidden-layer multivariate ReLU network trained with SGD · MNIST 0-vs-1 | predictor smoothness | Helped |
| Increase the step size | Single-hidden-layer multivariate ReLU network trained with GD · Synthetic regression | bias-vector norm | Helped |
| Use two hidden layers instead of one | ReLU network implementing p(x) = σ(1 − ||x||₁) · Stable approximation of p | stable representability | Helped |
| Restrict the target to sufficiently smooth Sobolev functions | Single-hidden-layer ReLU networks trained with GD · Compact subsets of R^d | stable approximation | Helped |
| Use the stability-norm lower bound to predict sharpness | Single-hidden-layer ReLU network trained with GD · Synthetic regression | sharpness-bound tightness | Helped |
Closest papers by meaning
- The Implicit Bias of Minima Stability: A View from Function Space (NeurIPS 2021 · similarity 0.94)
- Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes (NeurIPS 2024 · similarity 0.93)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.93)
- On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias (NeurIPS 2022 · similarity 0.92)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.89)
- Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.