Research map / Learning theory and algorithms
Kernels and regression theory: research map
1,004 accepted papers on Kernels and regression theory in Learning theory and algorithms, from ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), grouped into 3 clusters and 14 approaches. The busiest year so far is 2026.
Within Learning theory and algorithms, its share held steady from 11.1% in 2023–24 to 10.2% in 2025–26 (248 → 247 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).
Explore Kernels and regression theory in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
Approaches and key papers
kernels · hilbert · reproducing · 391 papers
Approaches in this cluster:
- RKHS kernel methods (127 papers)
Analyze and accelerate kernel methods with Hilbert space theory, Nystrom and kernel thinning. - Neural tangent kernel analysis (109 papers)
Study wide networks via the neural tangent kernel and feature-learning limits. - Kernel regression under shift (89 papers)
Provide consistency and learning guarantees for kernel estimators in varied settings. - Kernel two-sample testing (30 papers)
Learn and combine kernels for MMD-based two-sample and independence tests. - Random feature approximation (36 papers)
Approximate kernels with random Fourier and orthogonal features, with error bounds.
Most cited and most cited since 2024:
- Exponential expressivity in deep neural networks through transient chaos (NeurIPS 2016 · 290 citations)
- Object Tracking via Dual Linear Structured SVM and Explicit Feature Map (CVPR 2016 · 264 citations)
- PINNACLE: PINN Adaptive ColLocation and Experimental points selection (ICLR 2024 · 6 citations)
- Error Bounds for Gaussian Process Regression Under Bounded Support Noise with Applications to Safety Certification (AAAI 2025 · 5 citations)
generalization bounds · pac · generalization error · 360 papers
Approaches in this cluster:
- Statistical perspectives on generalization (99 papers)
Analyze generalization via invariance, interpolation, shift and information-theoretic views of data. - Information-theoretic generalization bounds (94 papers)
Bound generalization error with mutual information and conditional mutual information. - Generalization theory of deep networks (67 papers)
Study generalization of ReLU, ResNet and deep architectures via width, depth and sparsity. - Empirical generalization measures (61 papers)
Measure and predict generalization of deep networks through experiments and scaling laws. - PAC-Bayes bounds (39 papers)
Derive PAC-Bayesian generalization bounds with data-dependent priors and for meta-learning.
Most cited and most cited since 2024:
- Exploring Generalization in Deep Learning (NeurIPS 2017 · 450 citations)
- A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (ICLR 2018 · 296 citations)
- Improving Neural Network Generalization on Data-Limited Regression with Doubly-Robust Boosting (AAAI 2024 · 12 citations)
- A Unified Recipe for Deriving (Time-Uniform) PAC-Bayes Bounds (NeurIPS 2024 · 6 citations)
ridge · linear regression · regularization · 253 papers
Approaches in this cluster:
- High-dimensional asymptotic generalization (82 papers)
Derive exact asymptotic error of models in proportional high-dimensional limits using random matrix tools. - High-dimensional linear regression theory (61 papers)
Analyze risk of least-squares, interpolators and SVR in high dimensions. - Double descent and overfitting (53 papers)
Explain benign and harmful overfitting and double descent in overparameterized models. - Kernel ridge regression analysis (57 papers)
Characterize learning curves and test error of kernel ridge regression.
Most cited and most cited since 2024:
- Generalization Properties of Learning with Random Features (NeurIPS 2017 · 184 citations)
- On the Spectral Bias of Neural Networks (ICML 2019 · 151 citations)
- A Statistical Theory of Regularization-Based Continual Learning (ICML 2024 · 7 citations)
- Model Collapse Demystified: The Case of Regression (NeurIPS 2024 · 6 citations)
Related topics in Learning theory and algorithms
- Matrix and tensor methods (1,074)
- Sample complexity (1,363)
- Combinatorial and search optimization (1,492)
- Fairness (516)
- Submodular and game algorithms (1,030)
- Prediction and decision losses (2,134)
- Clustering (725)
- Optimal transport (384)