Research map / Probabilistic and scientific modeling / Bayesian optimization

Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds

NeurIPS 2026 · Probabilistic and scientific modeling › Bayesian optimization · approach: Acquisition function design

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes L0MO, a Bayesian optimization method over functions in an RKHS that jointly optimizes sparse kernel locations and coefficients, and releases a functional benchmark suite built from ridge functionals.

Problem

Bayesian optimization of black-box functionals whose input is an infinite-dimensional function (functional BO), where prior methods restrict search to fixed or random linear subspaces.

Approach

Search the set of RKHS functions with at most m RBF kernel terms, optimizing both support locations and coefficients via a pullback-metric natural gradient view and multi-start L-BFGS-B, and show prior kernel-gradient FBO updates are natural gradient descent in a fixed-support parameterization.

Key results

• L0MO achieves lowest median average regret on Sinc-1D (1.93e-2 vs 2.58e-2 for Vien) and on all five ridge functionals, e.g. Rosenbrock average regret 3.04e4 vs 5.39e4 for Vien.
• On real tasks L0MO has lowest median on Pendulum, Pinwheel and Hopper, but Shilton reaches slightly lower final regret on Brachistochrone (0.87e-2 vs 1.02e-2).
• Under a uniform Matern-5/2 kernel, L0MO is tied for first on every benchmark except Brachistochrone final value; mean rank 1.00 avg and 1.10 final.

Surprising findings

• Shilton reaches a lower final regret than L0MO on Brachistochrone, attributed to local refinement of a smooth non-sparse optimum.
• The sparsity-based L0MO is not uniformly best: Vien has marginally lower final value on Sinc-4D and Sinc-5D.

Limitations

• No convergence or regret guarantee; density only ensures good sparse approximations exist as m grows.
• Acquisition landscape is non-convex, so multi-start L-BFGS-B may miss global optimum.
• Cost of acquisition grows with support points and observations, limiting moderate evaluation budgets.

Experimental setup

Datasets: MNIST Sinc test function GramacyLee Ackley Hartmann Rosenbrock Michalewicz Brachistochrone OpenAI Gym pendulum Pinwheel robot task Hopper

Benchmarks: Sinc-1D to 5D projections GramacyLee Ackley Hartmann Rosenbrock Michalewicz Brachistochrone MNIST activation Pendulum Pinwheel Hopper

Metrics: median average regret median final regret 95% CI

Baselines: Random Vien Kundu Shilton Vellanki

Builds on: Vien functional BO Kundu subspace BO Shilton sequential BO Vellanki Bernstein BO kernel matching pursuit

Closest papers by meaning

• Thompson Sampling in Function Spaces via Neural Operators (NeurIPS 2025 · similarity 0.87)
• Joint Composite Latent Space Bayesian Optimization (ICML 2024 · similarity 0.87)
• Kernelized Normalizing Constant Estimation: Bridging Bayesian Quadrature and Bayesian Optimization (AAAI 2024 · similarity 0.86)
• Vanilla Bayesian Optimization Performs Great in High Dimensions (ICML 2024 · similarity 0.86)
• Random Function Descent (NeurIPS 2024 · similarity 0.86)
• A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.86)