Research map / Optimization / Implicit bias and generalization
When Do Neural Networks Outperform Kernel Methods?
Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz, Andrea Montanari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Randomize 1.5% of low-frequency components | Myrtle-5 CNN and Myrtle-5 CNTK KRR · CIFAR-10 | classification accuracy: 87.7% -> 84.7%; 77.6% -> 58.2% | Mixed |
| Increase high-frequency noise in the training and test images | Neural networks and RKHS methods · FMNIST | classification accuracy | Mixed |
| Increase high-frequency noise in the training and test images | Neural networks and RKHS methods · CIFAR-2 | classification accuracy gap between NN and NT KRR: 0.6% at tau = 0 -> 4.5% at tau = 3 | Improved |
| Replace low-frequency image components with Gaussian noise | Neural networks and RKHS methods · FMNIST | classification accuracy | Mixed |
| Replace low-frequency image components with Gaussian noise | Myrtle-5 CNN and CNTK KRR · CIFAR-10 | classification accuracy | Mixed |
Closest papers by meaning
- Gradient Descent in Neural Networks as Sequential Learning in Reproducing Kernel Banach Space (ICML 2023 · similarity 0.86)
- Finite Versus Infinite Neural Networks: an Empirical Study (NeurIPS 2020 · similarity 0.86)
- Neural Networks Efficiently Learn Low-Dimensional Representations with SGD (ICLR 2023 · similarity 0.85)
- On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory (NeurIPS 2024 · similarity 0.85)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.84)
- Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
