Research map / Learning theory and algorithms / Kernels and regression theory
Kernel Quantile Embeddings and Associated Probability Metrics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces kernel quantile embeddings (directional quantiles in an RKHS) and kernel quantile discrepancies, showing they are probability metrics under weaker kernel conditions than MMD and yielding a near-linear Gaussian e-KQD two-sample test.
Problem
Kernel mean embeddings and MMD represent distributions only through mean functions in an RKHS; it is unclear whether other RKHS representations (quantiles) can give probability metrics under weaker kernel conditions with efficient estimators.
Approach
Define α-directional quantiles of the kernel feature map k(X,·) along unit RKHS directions, aggregate their differences over quantile levels and directions to get expected and supremum kernel quantile discrepancies; sample Gaussian-measure directions via a Monte Carlo approximation so that the estimator costs O(n log^2 n).
Key results
- Laplace vs Gaussian experiment with degree-3 polynomial kernel: KQD tests gain power with sample size while MMD-based tests fail to detect any difference, since the kernel matches first two moments.
- On CIFAR-10 vs CIFAR-10.1, near-linear KQD estimators significantly outperform the near-linear MMD-Multi estimator of the same complexity.
- Quadratic-time quantile-based estimators perform comparably to quadratic-time MMD estimators, and e-KQD shows the slowest power decline with dimension in the power-decay experiment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use e-KQD instead of MMD-Multi | Two-sample testing · Power-decay experiment (Gaussian distributions; dimensions 32–512) | rejection rate | Helped |
| Use KQDs instead of MMD-based tests with a degree-3 polynomial kernel | Two-sample testing · Gaussian vs. Laplace | rejection rate | Helped |
| Add the e-KQD term to the centered estimator (ablation) | e-KQD-centered · Gaussian vs. Laplace | rejection rate | Helped |
| Use e-KQD or sup-KQD instead of MMD-Multi | Near-linear two-sample tests · Galaxy MNIST | rejection rate | Helped |
| Use e-KQD or sup-KQD instead of fast MMD estimators | Near-linear two-sample tests · CIFAR-10 vs. CIFAR-10.1 | rejection rate | Helped |
| Use e-KQD-centered instead of MMD (ablation) | Quadratic-time two-sample tests · Galaxy MNIST | rejection rate | No clear effect |
| Set the KQD power to p=1 instead of p=2 (ablation) | KQD-based discrepancies · Two-sample testing experiments | rejection rate | No clear effect |
Surprising findings
- Every mean-characteristic kernel is quantile-characteristic, yet the converse fails, so quantile-based discrepancies can distinguish distributions that MMD cannot.
- The centered e-KQD variant behaves nearly identically to MMD on Galaxy MNIST, suggesting its MMD term dominates.
Limitations
- sup-KQD is approximated by maximising over sampled projections, so it is not an accurate estimate of the true supremum distance
- Centered e-KQD is dominated by the MMD term in practice, offering little gain on real data
- Gaussian measure sampling requires a reference measure choice and the uniform weighting nu is a user choice
Experimental setup
Datasets: Galaxy MNIST CIFAR-10 CIFAR-10.1 synthetic Gaussian and Laplace samples
Benchmarks: power-decay synthetic two-sample test Laplace vs Gaussian two-sample test Galaxy MNIST two-sample test CIFAR-10 vs CIFAR-10.1 two-sample test
Metrics: rejection rate (test power) Type I error runtime in seconds
Baselines: MMD MMD-Multi MMD-Lin sup-KQD e-KQD-centered
Builds on: maximum mean discrepancy sliced Wasserstein distance generalised sliced Wasserstein Sinkhorn divergence MMD-Multi
Closest papers by meaning
- On Statistical Learning Theory for Distributional Inputs (ICML 2024 · similarity 0.88)
- Statistical and Geometrical properties of the Kernel Kullback-Leibler divergence (NeurIPS 2024 · similarity 0.88)
- Learning to Embed Distributions via Maximum Kernel Entropy (NeurIPS 2024 · similarity 0.88)
- On the Consistency of Kernel Methods with Dependent Observations (ICML 2024 · similarity 0.86)
- Conditional Distribution Compression via the Kernel Conditional Mean Embedding (NeurIPS 2025 · similarity 0.86)
- Statistical and Computational Guarantees of Kernel Max-Sliced Wasserstein Distances (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.