Research map / Probabilistic and scientific modeling / Bayesian optimization

Kernel-based guarantees for nonlinear parametric models in Bayesian optimization

NeurIPS 2026 · Probabilistic and scientific modeling › Bayesian optimization · approach: Local and high-dimensional Bayesian optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Develops an RKHS framework over parameter-space kernels to give confidence bounds for regularized nonlinear parametric models trained on adaptive data, and derives a finite-domain approximate Thompson sampling regret bound for randomized regularized models.

Problem

Theoretical guarantees for nonlinear parametric models (e.g., finite-width neural networks) used as surrogates or acquisition models in Bayesian optimization under adaptive data collection are limited; existing analyses cover GPs, kernel machines, linear models, or linearized networks.

Approach

Embed the parametric model class into a model-induced RKHS via a parameter-space kernel, so that regularized losses are strongly convex in that RKHS; bound pointwise approximation error by GP posterior standard deviation using a Matérn domination of the model kernel. Randomized models regularized toward a fresh random initialization are maximized greedily, acting as approximate Thompson sampling.

Key results

• Confidence bound: |g(x,theta_t)-g(x,theta*)| <= 2 beta_t(delta) sigma_t(x) with GP posterior std under a model-agnostic kernel k dominating k_G, holding with probability at least 1-delta.
• Finite-domain approximate Thompson sampling with lambda_t ~ log^q t (q>1) achieves cumulative regret O(sqrt(T) (log T)^{zeta + q(1+f/4)}) with high probability under realizability, unique optimum, and small-ball condition exponent f.
• Posterior variance at the optimum satisfies sigma_t^2(x*) = O((log t)^{q f/2}/t) when x* is visited often enough.

Limitations

• Regret theorem limited to finite domains, realizable settings, and unique optimum for approximate Thompson sampling.
• General LFBO acquisitions with time-varying thresholds not covered by the formal regret analysis.
• Regret bound has an unresolved dependence on the small-ball exponent f in the RKHS pseudometric; no experiments reported.

Experimental setup

Metrics: cumulative regret posterior-style confidence width

Baselines: GP-UCB GP Thompson sampling BORE

Builds on: likelihood-free Bayesian optimization BORE neural Thompson sampling randomize-then-optimize self-normalized martingale bounds (Abbasi-Yadkori et al.)

Closest papers by meaning

• Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs (NeurIPS 2025 · similarity 0.88)
• Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization (NeurIPS 2026 · similarity 0.88)
• Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.87)
• BARK: A Fully Bayesian Tree Kernel for Black-box Optimization (ICML 2025 · similarity 0.87)
• Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiency (ICML 2024 · similarity 0.87)
• Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.86)