Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Domain Invariant Learning for Gaussian Processes and Bayesian Exploration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DIL-GP, a min-max domain-invariant learning method for Gaussian processes that infers worst-case data partitions (soft two-environment labels) and applies an IRM penalty, then uses it as a Bayesian optimization surrogate for quadrotor PID tuning.
Problem
Gaussian processes degrade under out-of-distribution (OOD) shifts, especially in small-data regimes, and standard domain-generalization methods need manually provided domain labels that are unavailable in many GP and Bayesian optimization settings.
Approach
Infer latent domains by adversarially partitioning the data (soft two-environment indicator via sigmoid) to maximize an IRM invariance penalty on the GP likelihood, then minimize likelihood plus that penalty over kernel parameters; the resulting DIL-GP is used as a surrogate in Bayesian optimization.
Key results
- On the 1-D synthetic dataset, DIL-GP achieves RMSE 0.3415 ± 0.0109 versus 0.3923 for vanilla GP and 0.3558 for random forest.
- On the King Housing domain4 test split, DIL-GP reaches RMSE 0.543 ± 0.058 versus 0.989 for vanilla GP.
- In quadrotor PID tuning, DIL-GP-based Bayesian optimization attains lowest trajectory errors on all four trajectories (e.g., Hover 0.3610 vs 0.3739 for GP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply domain-invariant learning to the Gaussian-process surrogate | GP · 1-Dim Dataset | RMSE: 0.3923 $\pm$ 0.0153 -> 0.3415 $\pm$ 0.0109 | Helped |
| Apply domain-invariant learning to the Gaussian-process surrogate | GP · 2-Dim Dataset | RMSE: 0.8022 $\pm$ 0.1373 -> 0.6583 $\pm$ 0.0230 | Helped |
| Apply domain-invariant learning to the Gaussian-process surrogate | GP · King Housing domains 1–4 | RMSE: 0.371 $\pm$ 0.009 -> 0.291 $\pm$ 0.016; 0.594 $\pm$ 0.056 -> 0.371 $\pm$ 0.011; 0.800 $\pm$ 0.084 -> 0.489 $\pm$ 0.042; | Helped |
| Apply domain-invariant learning to the Gaussian-process surrogate | GP · Automobile test dataset | RMSE: 0.889 $\pm$ 0.008 -> 0.837 $\pm$ 0.069 | Helped |
| Use DIL-GP instead of a vanilla GP surrogate in Bayesian optimization | GP Bayesian optimization · Hover quadrotor trajectory | Trajectory error: 0.3739$\pm$0.0187 -> 0.3610 $\pm$ 0.0200 | Helped |
| Use DIL-GP instead of a vanilla GP surrogate in Bayesian optimization | GP Bayesian optimization · Fig-8 quadrotor trajectory | Trajectory error: 0.4433$\pm$0.0241 -> 0.4081 $\pm$ 0.0262 | Helped |
| Use DIL-GP instead of a vanilla GP surrogate in Bayesian optimization | GP Bayesian optimization · Sin-forward quadrotor trajectory | Trajectory error: 0.3579$\pm$0.0032 -> 0.3367 $\pm$ 0.0043 | Helped |
| Use DIL-GP instead of a vanilla GP surrogate in Bayesian optimization | GP Bayesian optimization · Spiral-up quadrotor trajectory | Trajectory error: 0.3727$\pm$0.0333 -> 0.3349 $\pm$ 0.0275 | Helped |
| Switch the optimizer from SGD to Adam or LBFGS (ablation) | DIL-GP · Experiments | Performance | No clear effect |
Surprising findings
- The GP-DP kernel fails to fit the 1-D synthetic data (RMSE 1.04, coverage 0), showing kernel changes do not generalize across domains.
- MLP outperforms DIL-GP on some housing domains but is significantly unstable.
Limitations
- Fixes the number of latent environments to two.
- Gains are modest or unstable on some real-world splits (MLP better on some housing domains).
- Theoretical OOD bound relies on strong assumptions on eigenvalue decay and an idealized environment basis.
Experimental setup
Datasets: King County house sales (Kaggle) UCI Automobile dataset synthetic 1-D and 2-D clustered datasets Dryden wind model quadrotor simulation
Benchmarks: synthetic 1-D dataset synthetic 2-D dataset King County housing UCI Automobile quadrotor PID tuning trajectories (Hover, Fig-8, Sin-forward, Spiral-up)
Metrics: RMSE coverage rate averaged control error
Baselines: Gaussian process (RBF) GP with rational quadratic kernel GP with dot product kernel random forest multi-layer perceptron
Builds on: invariant risk minimization Gaussian processes upper confidence bound Bayesian optimization
Closest papers by meaning
- Sample-efficient Bayesian Optimisation Using Known Invariances (NeurIPS 2024 · similarity 0.87)
- Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference (NeurIPS 2024 · similarity 0.87)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.87)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.86)
- Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · similarity 0.86)
- Robust and Computation-Aware Gaussian Processes (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.