Research map / Probabilistic and scientific modeling / Bayesian optimization

BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability

NeurIPS 2026 · Probabilistic and scientific modeling › Bayesian optimization · approach: Acquisition function design · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BONSAI, a default-aware Bayesian optimization policy that greedily reverts low-impact parameter deviations from a default configuration while bounding the acquisition loss, with GP-UCB regret bounds and empirical sparsity-performance gains.

Problem

Standard Bayesian optimization ignores a default configuration and tends to change many weakly relevant parameters, pushing them to boundaries, which makes recommendations costly to vet and deploy.

Approach

Apply a greedy acquisition-gap-constrained pruning step to the acquisition maximizer, resetting coordinates to default whenever the relative loss in (baseline-shifted) acquisition value stays under a threshold; theory shows GP-UCB regret plus a threshold-dependent penalty and sparsity recovery under idealized ARD assumptions.

Key results

• Candidate-generation cost of BONSAI averages about 1.5x that of standard BO versus 7-34x for prior sparse-BO methods.
• BONSAI succeeds in 20/20, 16/20, 20/20, and 20/20 replications on Joint NAS/HPO, LassoDNA, Cell Network, and Optical Design, while PiBO produces no epsilon-feasible point on Joint NAS/HPO, LassoDNA, or Cell Network.
• On Optical Design BONSAI finds excellent Pareto frontiers using an average of fewer than 60 active dimensions (<50% of parameters).

Surprising findings

• PiBO's soft prior produces no epsilon-feasible point in any of 20 replications on several real-world problems despite biasing toward the default.
• Greedy single-coordinate pruning matched exact pruning in performance and returned the same first post-initialization candidate in 85.7% of runs on low-dimensional problems.

Limitations

• Sparsity is defined relative to the supplied default, so default placement can change apparent sparsity and advantage.
• Sparsity recovery theory relies on idealized perfect ARD lengthscale separation and gap assumptions; regret theory is specific to GP-UCB, not EI.
• Greedy single-coordinate pruning can miss joint-reset configurations and does not establish minimum cardinality.

Experimental setup

Datasets: Branin Hartmann PressureVessel DTLZ2 Optical Design LassoDNA Cell Network Joint NAS/HPO

Benchmarks: Branin (embedded 50d) Hartmann (embedded 50d) PressureVessel (embedded 50d) DTLZ2 (embedded 50d) Joint NAS/HPO Optical Design LassoDNA Cell Network

Metrics: best objective hypervolume number of active dimensions candidate generation time feasibility count at epsilon=0.2

Baselines: Sobol sampling Vanilla BO (EI) IR ER SEBO PiBO

Builds on: GP-UCB Expected Improvement SEBO PiBO qLogNEI

Closest papers by meaning

• Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.90)
• Bayesian Optimization for Unknown Cost-Varying Variable Subsets with No-Regret Costs (AAAI 2025 · similarity 0.89)
• A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.89)
• Cost-aware Bayesian Optimization via the Pandora's Box Gittins Index (NeurIPS 2024 · similarity 0.89)
• Posterior Sampling-Based Bayesian Optimization with Tighter Bayesian Regret Bounds (ICML 2024 · similarity 0.89)
• FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch (ICML 2025 · similarity 0.88)