Research map / Learning theory and algorithms / Kernels and regression theory
From Tempered to Benign Overfitting in ReLU Neural Networks
Guy Kornowski, Gilad Yehudai, Ohad Shamir
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase input dimension d relative to sample size m (ablation) | 2-layer ReLU net (m=500, 2000) · Uniform sphere data with label noise p, 2-layer ReLU net | clean test error | Improved |
| vary width between 50 and 3000 neurons (ablation) | 2-layer ReLU net (m=500, d in 250, 2000, 10000) · Uniform sphere data with label noise p, 2-layer ReLU net | clean test error | No clear effect |
| use 3-layer network instead of 2-layer (ablation) | 3-layer ReLU net with 1000 neurons, m=500 · Uniform sphere data with label noise p, 2-layer ReLU net | clean test error | No clear effect |
| raise label noise p toward 0.5 at high dimension (ablation) | 2-layer ReLU net · Uniform sphere data with label noise p, 2-layer ReLU net | clean test error | Hurt |
| train for 100k instead of 20k epochs (ablation) | 2-layer ReLU net · Uniform sphere data with label noise p, 2-layer ReLU net | clean test error | No clear effect |
Closest papers by meaning
- Noisy Interpolation Learning with Shallow Univariate ReLU Networks (ICLR 2024 · similarity 0.94)
- Benign overfitting in leaky ReLU networks with moderate input dimension (NeurIPS 2024 · similarity 0.92)
- Provable Tempered Overfitting of Minimal Nets and Typical Nets (NeurIPS 2024 · similarity 0.91)
- Rethinking Benign Overfitting in Two-Layer Neural Networks (ICML 2025 · similarity 0.91)
- Benign Overfitting in Adversarial Training of Neural Networks (ICML 2024 · similarity 0.90)
- Simplicity Bias and Optimization Threshold in Two-Layer ReLU Networks (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
