Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
PAC-Bayes and decoupled GP learning
Learns Gaussian processes via decoupled variational bases, PAC-Bayes bounds and random projections.
83 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Uncertainty and Gaussian processes, keyword cluster “gaussian processes · gp · kernel”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
16 papers in 2023–24 and 18 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 5.9% to 5.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Deep Gaussian Processes for Regression using Approximate Expectation Propagation (ICML 2016 · 125 citations)
- 2017 · Spectral Mixture Kernels for Multi-Output Gaussian Processes (NeurIPS 2017 · 56 citations)
- 2018 · Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo (NeurIPS 2018 · 63 citations)
- 2019 · Robustness Guarantees for Bayesian Inference with Gaussian Processes (AAAI 2019 · 39 citations)
- 2020 · Matérn Gaussian Processes on Riemannian Manifolds (NeurIPS 2020 · 25 citations)
- 2021 · Practical and Rigorous Uncertainty Bounds for Gaussian Process Regression (AAAI 2021 · 48 citations)
- 2022 · Bayesian Active Learning with Fully Bayesian Gaussian Processes (NeurIPS 2022 · 9 citations)
- 2023 · Gaussian processes at the Helm(holtz): A more fluid model for ocean currents (ICML 2023 · 3 citations)
- 2024 · Efficient Nonparametric Tensor Decomposition for Binary and Count Data (AAAI 2024 · 4 citations)
- 2025 · Asynchronous Distributed Gaussian Process Regression (AAAI 2025 · 3 citations)
- 2026 · Streaming Generated Gaussian Process Experts for Online Learning and Control (AAAI 2026 · 1 citation)
Earliest papers here
- Deep Gaussian Processes for Regression using Approximate Expectation Propagation (ICML 2016 · 125 citations)
- Gaussian Processes for Survival Analysis (NeurIPS 2016 · 32 citations)
- Incremental Variational Sparse Gaussian Process Regression (NeurIPS 2016 · 24 citations)
What has been tried
70 papers here report 657 measured changes (a component added, removed or swapped, compared with a baseline): helped 61%, no clear effect 14%, hurt 19%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase the inducing-point count | Article, rank 10 · RMSE | Hurt | Efficient Nonparametric Tensor Decomposition for Binary and Count Data (AAAI 2024 · 4 citations) | |
| Use the forward relevance-pursuit variant instead of the backward variant | Regression with few outliers · performance | Helped | Robust Gaussian Processes via Relevance Pursuit (NeurIPS 2024 · 4 citations) | |
| Increase the information set threshold from ¯I = 4 to ¯I = 10 | KIN40K · SMSE | Helped | Asynchronous Distributed Gaussian Process Regression (AAAI 2025 · 3 citations) | |
| Use the asymmetric e+r SVGP pair rather than a single-branch pairing | CoLA · MCC | 28.71 -> 29.31; 28.10 -> 29.31 | Helped | Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes (ICML 2024 · 1 citation) |
| Remove the shrinkage term | Synthetic dataset with 10% uniformly generated outliers · predictive posterior mean | Hurt | Robust and Conjugate Gaussian Process Regression (ICML 2024 · 1 citation) | |
| Use finer summarized-data grids | California housing, MedInc, Laplacian covariance · normalized posterior-mean RMSE | 0.57 ± 0.04 -> 0.32 ± 0.05 | Hurt | Learning from Summarized Data: Gaussian Process Regression with Sample Quasi-Likelihood (AAAI 2025 · 0 citations) |
| replace the subspace preconditioner with the identity matrix | large-scale regression datasets · RMSE | 0.795 -> 0.808 (yolanda); 0.752 -> 0.782 (song); 0.012 -> 0.168 (benzene); 0.015 -> 0.231 (malonaldehyde); 0.789 -> 0.79 | Hurt | Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · 0 citations) |
| increase latent dimension from 128 to 256 | AOD · RMSE | Helped | STACI: Spatio-Temporal Aleatoric Conformal Inference (NeurIPS 2025 · 0 citations) | |
| Set soft-threshold quantile parameter ε to 0.1 or 0.15 | UCI regression · MAE and NLL | Helped | Robust and Computation-Aware Gaussian Processes (NeurIPS 2025 · 0 citations) | |
| Use deep ensembles instead of variational inference for uncertainty quantification | Synthetic MSS interpolation · CRPS | 0.071 ± 0.019 -> 0.046 ± 0.005 | Helped | Deep Random Features for Scalable Interpolation of Spatiotemporal Data (ICLR 2025 · 0 citations) |
| Use an amortization grid that is too small (N = 5) | Surface reconstruction · reconstruction fidelity | Hurt | Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes (ICML 2025) |
What reviewers will expect
Datasets and baselines named in the 13 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- SARCOS · 2 of 13 papers
Baselines
- SVGP · 4 of 13 papers
- CaGP · 2 of 13 papers
- GPSat · 2 of 13 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 34 papers; from OpenAlex and the papers' affiliations).
- University of Oxford · 3 papers
- Aalto University · 3 papers
- Massachusetts Institute of Technology · 2 papers
- EPFL · 2 papers
- University College London · 2 papers
- University of Cambridge · 2 papers
- Meta · 2 papers
- Columbia University · 2 papers
- Cornell University · 2 papers
- ETHZ - ETH Zurich · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Sparse variational Gaussian processes (92 papers)
- Infinite-width networks as Gaussian processes (49 papers)
- Structured Gaussian process models (39 papers)
Elsewhere in the topic
- Nonparametric and PAC-Bayes Bayesian learning (130 papers)
- Simulation-based inference (118 papers)
- Priors and functional variational uncertainty (112 papers)
- Variational Bayesian deep learning (105 papers)
- Approximate inference in Bayesian neural networks (93 papers)
- Epistemic versus aleatoric decomposition (75 papers)
- Single-model predictive uncertainty (74 papers)
- Bayesian view of meta-learning (63 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).