Research map / Learning theory and algorithms / Kernels and regression theory
Kernel regression under shift
Provide consistency and learning guarantees for kernel estimators in varied settings.
89 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Kernels and regression theory, keyword cluster “kernels · hilbert · reproducing”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
19 papers in 2023–24 and 25 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 7.7% to 10.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Object Tracking via Dual Linear Structured SVM and Explicit Feature Map (CVPR 2016 · 264 citations)
- 2017 · Product Manifold Filter: Non-Rigid Shape Correspondence via Kernel Density Estimation in the Product Space (CVPR 2017 · 120 citations)
- 2018 · Kernel Cross-Correlator (AAAI 2018 · 116 citations)
- 2019 · Fast Incremental SVDD Learning Algorithm with the Gaussian Kernel (AAAI 2019 · 74 citations)
- 2020 · Minimax Estimation of Conditional Moment Models (NeurIPS 2020 · 21 citations)
- 2021 · Meta-Learning with Neural Tangent Kernels (ICLR 2021 · 4 citations)
- 2022 · Efficient Dataset Distillation using Random Feature Approximation (NeurIPS 2022 · 17 citations)
- 2023 · Practical Markov Boundary Learning without Strong Assumptions (AAAI 2023 · 12 citations)
- 2024 · On Hypothesis Transfer Learning of Functional Linear Models (ICML 2024 · 4 citations)
- 2025 · Error Bounds for Gaussian Process Regression Under Bounded Support Noise with Applications to Safety Certification (AAAI 2025 · 5 citations)
- 2026 · Conditional KRR: Injecting Unpenalized Features into Kernel Methods with Applications to Kernel Thresholding (ICML 2026 · 0 citations)
Earliest papers here
- Object Tracking via Dual Linear Structured SVM and Explicit Feature Map (CVPR 2016 · 264 citations)
- Persistence weighted Gaussian kernel for topological data analysis (ICML 2016 · 115 citations)
- Nonparametric Canonical Correlation Analysis (ICML 2016 · 68 citations)
What has been tried
66 papers here report 549 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 11%, hurt 19%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use a reflected Gaussian kernel instead of a standard Gaussian kernel | Predictions on [0,1] · boundary density bias | approximately factor of 2 | Helped | Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing (ICLR 2024 · 2 citations) |
| use a subgroup-invariant kernel rather than a standard kernel | PermInv-6D · sample efficiency | Helped | Sample-efficient Bayesian Optimisation Using Known Invariances (NeurIPS 2024 · 2 citations) | |
| Increase the number of RFF features | MS-COCO · cluster separation quality | Helped | Towards an Explainable Comparison and Alignment of Feature Embeddings (ICML 2025 · 0 citations) | |
| Use PEAKS with output-layer weight prototypes instead of PEAKS-V with validation-set class means | Fine-tuning datasets; limited samples per class (e.g., ×1) · test accuracy | Hurt | PEAKS: Selecting Key Training Examples Incrementally via Prediction Error Anchored by Kernel Similarity (ICML 2025 · 0 citations) | |
| Use an overly small kernel lengthscale instead of the HSIC-optimized lengthscale | Test case 2, n=100 · adaptivity | Hurt | Scalable and adaptive prediction bands with kernel sum-of-squares (NeurIPS 2025 · 0 citations) | |
| Use KSVD instead of SVD on degree-2 polynomial features | Diabetes, Ionosphere, and Liver · AUROC | 0.6296 -> 0.7607; 0.7292 -> 0.8374; 0.7032 -> 0.7100 | Helped | Learning in Feature Spaces via Coupled Covariances: Asymmetric Kernel SVD and Nyström method (ICML 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 15 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MUTAG · 2 of 15 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 44 papers; from OpenAlex and the papers' affiliations).
- The Chinese University of Hong Kong · 3 papers
- University of Science and Technology of China · 2 papers
- Carnegie Mellon University · 2 papers
- Korea Advanced Institute of Science and Technology · 2 papers
- RWTH Aachen University · 2 papers
- Rheinisch Westfälische Technische Hochschule Aachen · 2 papers
- Amazon · 2 papers
- Pennsylvania State University · 2 papers
- NTT · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- RKHS kernel methods (127 papers)
- Neural tangent kernel analysis (109 papers)
- Kernel two-sample testing (30 papers)
- Random feature approximation (36 papers)
Elsewhere in the topic
- Statistical perspectives on generalization (99 papers)
- Information-theoretic generalization bounds (94 papers)
- High-dimensional asymptotic generalization (82 papers)
- Generalization theory of deep networks (67 papers)
- Empirical generalization measures (61 papers)
- High-dimensional linear regression theory (61 papers)
- Kernel ridge regression analysis (57 papers)
- Double descent and overfitting (53 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).