Research map / Learning theory and algorithms / Kernels and regression theory

On Statistical Learning Theory for Distributional Inputs

ICML 2024 · Learning theory and algorithms › Kernels and regression theory · approach: Kernel regression under shift

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Theoretical paper proving two oracle inequalities and an algorithmic-stability generalization bound for kernel SVMs on distributional inputs, formulated for general Hilbertian embeddings and specialized to kernel mean embeddings and sliced Wasserstein.

Problem

Statistical learning theory for kernel-based learning with distributional inputs has mostly covered distributional regression with kernel ridge regression; classification and general regularized risk minimization lack oracle inequalities and generalization bounds.

Approach

Generalize distributional learning analysis to arbitrary Hilbertian embeddings of probability distributions combined with regularized empirical risk minimization in RKHSs, deriving oracle inequalities (via Steinwart & Christmann proof strategies) and stability-based generalization bounds that propagate embedding estimation error through the loss.

Key results

• Proves two oracle inequalities for regularized risk minimization (SVMs) on distributional inputs under general Hilbertian embeddings, one via Lipschitz-type loss regularity (Theorem 3.1) and one via RKHS structure with covering numbers (Theorem 3.4).
• Derives a stability-based generalization bound for SVMs in the two-stage sampling setup (Theorem 4.2), with the embedding error term E||ΠQ - ΠS̃||, specialized to KMEs giving a sqrt(E[sqrt(M)]) rate term (Corollary 4.5).
• Specializes results to kernel mean embeddings and sliced Wasserstein Hilbertian embeddings, with the sliced 2-Wasserstein case deferred to the appendix.

Limitations

• Learning rates require additional assumptions to bound the approximation error, which are left open.
• Requires various boundedness assumptions on loss functions; removing them is open.
• Classic arguments used; sharper bounds via empirical process theory are left for future work.

Experimental setup

Builds on: Caponnetto & De Vito 2007 Meunier et al. 2022 Steinwart & Christmann 2008 Szabó et al. 2015 Bousquet & Elisseeff 2002

Closest papers by meaning

• Statistical Learning Theory for Distributional Classification (AAAI 2026 · similarity 0.93)
• Learning to Embed Distributions via Maximum Kernel Entropy (NeurIPS 2024 · similarity 0.89)
• Distribution Regression with Sliced Wasserstein Kernels (ICML 2022 · similarity 0.89)
• Kernel Quantile Embeddings and Associated Probability Metrics (ICML 2025 · similarity 0.88)
• On the Consistency of Kernel Methods with Dependent Observations (ICML 2024 · similarity 0.87)
• Generalization in Kernel Regression Under Realistic Assumptions (ICML 2024 · similarity 0.87)