Research map / Optimization / Implicit bias and generalization
Minimum norm interpolation by perceptra: Explicit regularization and implicit bias
Jiyoung Park, Ian Pelakh, Stephan Wojtowytsch
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weight decay 1e-5 (explicit regularization) (ablation) | Momentum-SGD · Radial data d=31 | best rescaling factor: 1/3.5 -> 1/3.8 | Mixed |
| add weight decay 1e-5 (ablation) | Adam · Radial data d=31 | radial standard deviation | No clear effect |
| add explicit regularization with Xavier init (ablation) | 1-hidden-layer ReLU net · Radial data | minimum-norm shape | No clear effect |
| use He initialization with explicit regularization (ablation) | 1-hidden-layer ReLU net · Radial data | solution shape and radial symmetry | Improved |
| raise initialization gain from small to alpha=5 (ablation) | ReLU net, all optimizers · 1D |x| interpolation | interpolant convexity | Hurt |
| raise initialization gain alpha (ablation) | ReLU net, all optimizers · 1D |x| interpolation | initial training speed | Improved |
| widen network from m=200 to m=1,000 (ablation) | ReLU net · 1D |x| interpolation | variation across runs | Improved |
| use Adam instead of SGD variants (ablation) | Adam · Radial data d=31 | weight decay regularizer value: 1,600 | Hurt |
| use Adam instead of Momentum-SGD (ablation) | Adam · Radial data | radial standard deviation: 0.1 -> 0.2 | Hurt |
| increase dataset size (ablation) | ReLU net m=12,000 · Radial data d=15 | fit to minimum-norm interpolant | Improved |
| increase dimension from 3 to 31 (ablation) | ReLU net · Radial data | non-negativity of solution | Hurt |
| use uniform Xavier instead of normal Xavier init (ablation) | ReLU net · Radial data d=31 | radial symmetry | No clear effect |
Closest papers by meaning
- Penalising the biases in norm regularisation enforces sparsity (NeurIPS 2023 · similarity 0.91)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.89)
- Minimum Norm Interpolation Meets The Local Theory of Banach Spaces (ICML 2024 · similarity 0.88)
- Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks (ICML 2025 · similarity 0.88)
- A New Neural Kernel Regime: The Inductive Bias of Multi-Task Learning (NeurIPS 2024 · similarity 0.87)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
