Research map / Learning theory and algorithms / Kernels and regression theory
Predicting Kernel Regression Learning Curves from Only Raw Data Statistics
Dhruva Karkada, Joseph Turnbull, Yuxi Liu, James B Simon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decrease Gaussian kernel width so level coefficients decay more slowly (ablation) | HEA for Gaussian kernel · Gaussian data / real image data | agreement with HEA eigenstructure | Hurt |
| decrease effective data dimension (ablation) | HEA for Laplace kernel and ReLU NTK · real image data | agreement with HEA eigenstructure | Hurt |
| decrease effective data dimension (ablation) | HEA for Gaussian kernel · real image data | agreement with HEA eigenstructure | No clear effect |
| use simpler, less Gaussian datasets (CIFAR -> SVHN -> MNIST -> tabular) (ablation) | HEA · kernel eigenstructure on image/tabular datasets | agreement with HEA eigenstructure | Hurt |
| re-orthogonalize sampled Hermite polynomials with Gram-Schmidt when estimating target coefficients | HEA learning-curve prediction · KRR learning curves on real datasets | target coefficient estimation | Improved |
| truncate the HEA at finite order (5 to 10) and use high effective dimension | HEA for Laplace and ReLU kernels · real image data | agreement with HEA eigenstructure | Improved |
| apply sample normalization or ZCA whitening to the data | HEA · image datasets | agreement with HEA eigenstructure | Improved |
| predict KRR learning curves with HEA eigenstructure plus eigenframework | KRR with Gaussian, Laplace, ReLU NNGP, NTK kernels · CIFAR-5m, SVHN, ImageNet | test risk vs sample size | Improved |
| use HEA eigenvalue to predict MLP training time for Hermite targets | three-layer ReLU MLP in feature-learning regime · Gaussian data and CIFAR-5m multi-Hermite targets | steps to reach online MSE <= 0.1 | Improved |
Closest papers by meaning
- A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression (NeurIPS 2024 · similarity 0.88)
- Generalization in Kernel Regression Under Realistic Assumptions (ICML 2024 · similarity 0.87)
- Learning with Exact Invariances in Polynomial Time (ICML 2025 · similarity 0.87)
- Spectrally Transformed Kernel Regression (ICLR 2024 · similarity 0.86)
- Kernel conditional tests from learning-theoretic bounds (NeurIPS 2025 · similarity 0.86)
- Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrum (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
