Research map / Learning theory and algorithms / Kernels and regression theory
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the kernel eigenspectrum to predict learning curves by spectral mode | Kernel regression and NTK · Uniform hypersphere synthetic data | mode-wise generalization error | Helped |
| Increase the training set size | Dot-product kernels, including NTK · Uniform hypersphere synthetic data | mode-wise generalization error | Mixed |
| Use a kernel spectrum that decays more rapidly | Dot-product kernel regression · Uniform hypersphere synthetic data | frequency-dependent learning rate | Helped |
| Increase explicit regularization | NTK kernel regression · Synthetic data on the sphere | mode-wise generalization error | Hurt |
| Increase explicit regularization | Gaussian kernel regression · Standard normal data in dimension 20 | generalization error | Hurt |
| Train a finite-width neural network instead of using the infinite-width NTK prediction | Wide neural networks · Synthetic spherical-harmonic tasks | generalization error | No clear effect |
| Apply the NTK learning-curve theory to a discrete MNIST data measure | 3-layer NTK regression and neural network · MNIST subset of 8000 images | generalization error | No clear effect |
| Use the continuous approximation or replica saddle-point calculation | Kernel regression learning curves · General kernel regression | predicted generalization error | No clear effect |
| Increase the input dimension | 3-layer NTK kernel regression · Synthetic spherical-harmonic tasks | sample size needed to learn a frequency mode | Hurt |
| Increase the number of training samples across a frequency-mode threshold | Dot-product kernels · Uniform hypersphere, p proportional to d^l | mode-wise generalization error | Mixed |
Closest papers by meaning
- Generalization error of spectral algorithms (ICLR 2024 · similarity 0.90)
- Generalization in Kernel Regression Under Realistic Assumptions (ICML 2024 · similarity 0.89)
- On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory (NeurIPS 2024 · similarity 0.87)
- Sharp Generalization for Nonparametric Regression by Over-Parameterized Neural Networks: A Distribution-Free Analysis in Spherical Covariate (ICML 2025 · similarity 0.87)
- Neural Tangent Kernels Motivate Cross-Covariance Graphs in Neural Networks (ICML 2024 · similarity 0.87)
- Double-Descent Curves in Neural Networks: A New Perspective Using Gaussian Processes (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.