Research map / Learning theory and algorithms / Kernels and regression theory
Distributionally Robust Active Learning for Gaussian Process Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes two distributionally robust active learning methods for Gaussian process regression that minimize the worst-case expected posterior variance over an ambiguity set of target distributions, with convergence guarantees.
Problem
Active learning for GPR/kernel ridge regression lacks theoretical guarantees on prediction error under a target distribution, which is hard to specify; the paper targets worst-case expected squared error over a set of candidate target distributions.
Approach
Extends US/RS-style variance-based guarantees to distributionally robust AL: an RS variant samples from the worst-case target distribution, and an approximately greedy variant minimizes expected posterior variance under that distribution with a constraint on candidate uncertainty, yielding high-probability bounds on worst-case expected error.
Key results
- Proposed DR variance reduction and constrained DR variance reduction show consistently superior E_T across eta = 0, 0.001, 0.01, 0.1 on synthetic SE and Matern kernels and on the three real-world datasets, versus RS, US, variance reduction and EPIG.
- Theory gives E_T = O(log(|X|/delta) gamma_T / T) for discrete domains and O(log(T/delta) gamma_T / T) for continuous domains with linear, SE, and Matern kernels.
- EPIG, designed for eta=0, is inferior across all eta because it targets entropy rather than squared error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use distribution-aware DR or constrained DR variance reduction instead of standard active-learning baselines | Gaussian process regression active learning · Synthetic datasets | worst-case expected squared error E_T | Helped |
| Use distribution-aware DR or constrained DR variance reduction instead of standard active-learning baselines | Gaussian process regression active learning · King County house sales, red wine quality, and auto MPG | worst-case expected squared error E_T | Helped |
| Use uncertainty sampling or variance reduction without incorporating the ambiguity set | Gaussian process regression active learning · Synthetic datasets with small η | worst-case expected squared error E_T | Hurt |
| Use expected predictive information gain instead of an acquisition function targeting squared error | Gaussian process regression active learning · Synthetic datasets, η = 0, 0.001, 0.01, 0.1 | worst-case expected squared error E_T | Hurt |
| Remove the uncertainty constraint from constrained DR variance reduction, yielding DR variance reduction (ablation) | Constrained DR variance reduction · Synthetic datasets | worst-case expected squared error E_T | Helped |
| Use DR random sampling instead of DR or constrained DR variance reduction | Gaussian process regression active learning · Synthetic datasets | worst-case expected squared error E_T | Hurt |
Surprising findings
- The unconstrained greedy variant (DR variance reduction) performed better empirically than the theoretically constrained CDR variance reduction, though the constraint did not substantially hurt performance.
- Random-sampling-based DR random was often inferior to the greedy methods due to randomness.
Limitations
- Min-max optimization and expectation over the ambiguity set are computationally expensive; experiments restricted to discrete distributions under an L-infinity ball.
- Theoretical guarantee for the approximately greedy algorithm is not established; constraint on candidate set is needed for analysis only.
- GP computation is O(T^3) and analysis assumes exact GP inference.
Experimental setup
Datasets: King County house sales (1000-point subsample) red wine quality auto MPG synthetic GP sample paths on a 11^3 grid
Benchmarks: synthetic GP sample paths King County house sales red wine quality auto MPG
Metrics: worst-case expected squared error E_T max expected posterior variance
Baselines: random sampling uncertainty sampling variance reduction expected predictive information gain (EPIG) DR variance reduction (unconstrained ablation)
Builds on: kernelized bandits maximum information gain analysis distributionally robust learning Bayesian and frequentist GP confidence bounds transductive active learning
Closest papers by meaning
- Error Bounds for Gaussian Process Regression Under Bounded Support Noise with Applications to Safety Certification (AAAI 2025 · similarity 0.87)
- Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights (NeurIPS 2026 · similarity 0.86)
- Robust and Conjugate Gaussian Process Regression (ICML 2024 · similarity 0.84)
- Estimating Generalization Performance Along the Trajectory of Proximal SGD in Robust Regression (NeurIPS 2024 · similarity 0.84)
- Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes (NeurIPS 2024 · similarity 0.84)
- Robust and Computation-Aware Gaussian Processes (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.