Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Robust and Computation-Aware Gaussian Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RCaGP, a Gaussian process that combines robust generalized-Bayes conjugate updates (RCGP) with computation-aware low-rank inference (CaGP) to handle outliers and approximation uncertainty in regression and Bayesian optimization.
Problem
Sparse GPs used for regression and high-throughput Bayesian optimization are non-robust to outliers and overconfident where inducing points are far from data; existing fixes address only one of these issues.
Approach
Apply a probabilistic linear-solver treatment (computation-aware uncertainty) to the representer weights of a robust conjugate GP, with a robust weight function built on the prior mean, and optimize action matrices jointly with hyperparameters and query selection via EULBO.
Key results
- On UCI with asymmetric outliers, RCaGP achieves the lowest MAE on Boston (0.477), Energy (0.380) and Parkinsons (0.586) among the compared methods, and is beaten on Yacht only by RRP.
- On Lasso-DNA180D BO with EULBO-EI under asymmetric outliers, RCaGP clearly outperforms all baselines.
- The paper reports that the soft-threshold ablation is best around epsilon=0.15, matching the injected outlier fraction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine robust updating with computation-aware uncertainty (RCaGP versus RCSVGP) | UCI regression · Boston, asymmetric outliers | MAE: 0.532 ± 0.050 -> 0.477 ± 0.046 | Helped |
| Combine robust updating with computation-aware uncertainty (RCaGP versus CaGP) | UCI regression · Energy, asymmetric outliers | MAE: 0.562 ± 0.055 -> 0.380 ± 0.038 | Helped |
| Use RCaGP rather than RRP | UCI regression · Yacht, asymmetric outliers | MAE: 0.356 ± 0.051 -> 0.435 ± 0.052 | Hurt |
| Use RCaGP rather than CaGP | UCI regression · Energy, no outliers | MAE: 0.334 ± 0.024 -> 0.393 ± 0.024 | Hurt |
| Use RCaGP rather than CaGP | UCI regression · Boston, no outliers | MAE: 0.488 ± 0.042 -> 0.428 ± 0.053 | Helped |
| Set soft-threshold quantile parameter ε to 0.1 or 0.15 (ablation) | RCaGP weight function · UCI regression | MAE and NLL | Helped |
| Increase ε beyond 0.2 (ablation) | RCaGP weight function · UCI regression | MAE and NLL | Hurt |
| Use the expert-driven prior mean instead of a constant mean (ablation) | RCSVGP · Bayesian optimization tasks | best value found | Helped |
| Use the expert-driven prior mean instead of a constant mean (ablation) | RCaGP · Lunar12D | best value found | Helped |
Surprising findings
- Robustness alone (RCSVGP) with a different weight function did not match RCaGP, supporting that computational uncertainty and robustness must be combined.
- On the Energy dataset without outliers, CaGP marginally beats RCaGP in MAE.
Limitations
- Requires full batching, which increases GPU memory relative to stochastic SVGP.
- Projection dimensionality must be set manually and cannot be tuned via standard GP model selection.
- Evaluation limited to relatively simple regression and BO domains.
Experimental setup
Datasets: UCI Boston UCI Energy UCI Yacht UCI Parkinsons DJIA index April 17 2013 Hartmann6D Lunar12D Rover60D Lasso-DNA180D
Benchmarks: UCI regression (Boston, Energy, Yacht, Parkinsons) DJIA flash crash regression Hartmann6D Lunar12D Rover60D Lasso-DNA180D
Metrics: MAE NLL clock-time best value found in BO
Baselines: SVGP CaGP RCSVGP Student-t SVGP RRP TuRBO DPP-BO
Builds on: RCGP computation-aware GP SVGP EULBO RCSVGP
Closest papers by meaning
- Robust Gaussian Processes via Relevance Pursuit (NeurIPS 2024 · similarity 0.91)
- Robust and Conjugate Gaussian Process Regression (ICML 2024 · similarity 0.90)
- Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression (NeurIPS 2023 · similarity 0.90)
- Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference (NeurIPS 2024 · similarity 0.89)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.88)
- Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.