Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights

NeurIPS 2026 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Variational Bayesian deep learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formalizes active learning under a self-induced Boltzmann target distribution and proposes GP-based variance-reduction acquisitions (AB-SID-iVAR, TS-SID-iVAR) with high-probability and average-case convergence guarantees.

Problem

Active learning to minimize prediction error under a Boltzmann distribution whose energy is the unknown function itself, with an intractable partition function and unknown target distribution.

Approach

Approximates the expectation over the GP posterior of the Boltzmann target in closed form (zero-order Taylor expansion with Gaussian MGF identities), giving an unnormalized surrogate exp(λμ + λ²σ²/2 + b) used in integrated variance reduction with a potential set restricting queries to uncertain regions; a Thompson-sampling variant uses N=1 Monte Carlo.

Key results

• AB-SID-iVAR converges roughly an order of magnitude lower in weighted MSE than baselines on GuacaMol at λ=25, with a larger gap at λ=75.
• Theory gives high-probability terminal MSE rate O(T^{-1/2+ε}) with constant MC particles and O(T^{-1+ε}) when M=Ω(T).
• On GuacaMol top-k R² baselines degrade to strongly negative values as k shrinks while AB-SID-iVAR stays near zero.

Surprising findings

• Plug-in surrogates using the posterior mean alone (μ-SID) fail on multimodal (Branin) and oscillatory (Ishigami) targets, showing that the Bayesian treatment of uncertainty is necessary.
• Global metrics can look reasonable for SID-unaware methods while they fail on the high-weight subset relevant to downstream screening.

Limitations

• Theory relies on kernel regularity (four times differentiable stationary kernel) and bounded bias; GuacaMol experiments use a Tanimoto kernel outside that assumption.
• Continuous-domain implementation requires MCMC (SMC) and constrained acquisition optimization, increasing per-iteration cost (under 30 seconds).
• Average-case guarantee is in expectation over the GP prior, and the high-probability bound has an exponential prefactor.

Experimental setup

Datasets: GuacaMol

Benchmarks: GuacaMol Median 1 GuacaMol Median 2

Metrics: weighted MSE R^2 on top-k% compounds

Baselines: Random Sampling Uncertainty Sampling IMSE EPIG GHAL DRAL

Builds on: integrated mean squared error active learning distributionally robust active learning pathwise conditioning

Closest papers by meaning

• Distributionally Robust Active Learning for Gaussian Process Regression (ICML 2025 · similarity 0.86)
• Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes (NeurIPS 2024 · similarity 0.85)
• Stochastic Gradient Descent for Gaussian Processes Done Right (ICLR 2024 · similarity 0.85)
• BoltzNCE: Learning likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation (NeurIPS 2025 · similarity 0.84)
• Conditioning Gaussian Processes on Almost Anything (NeurIPS 2026 · similarity 0.84)
• Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project (NeurIPS 2025 · similarity 0.84)