Research map / Probabilistic and scientific modeling / Bayesian optimization
Boosting Offline Optimizers with Surrogate Sensitivity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BOSS, a model-agnostic sensitivity-informed regularizer that penalizes how often a surrogate's expected output changes under random parameter perturbations, and plugs it into 11 existing offline optimizers on Design-Bench tasks to improve their performance.
Problem
Offline optimization of black-box design functions relies on learned surrogates that are unreliable out of distribution, where they are sensitive to small parameter perturbations and can mislead the search.
Approach
Defines a model-agnostic (alpha, omega)-sensitivity measure of a surrogate as the probability that its dataset-averaged output changes by more than alpha under Gaussian weight perturbations, relaxes it into a differentiable upper bound with a neural re-parameterization for the perturbation distribution and a first-order Taylor approximation, and uses it as a minimax regularizer added to any offlin
Key results
- BOSS improves baseline performance in 56 of 66 cases at the 100th percentile, with average gain of about 2.08% and peak gain of 30% (REINFORCE on TF-Bind-8).
- CMA-ES on Ant Morphology gains 9.4% with BOSS, raising the reported normalized score from 1.915 to 2.009 (new SOTA claim).
- BOSS-2 (Taylor/CDF variant) gives smaller gains than BOSS; BOSS shows no degradation in the 18 comparison cases tested, unlike BOSS-2 which drops by 3.8% on REINFORCE Superconductor.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BOSS sensitivity regularization to offline optimizers | Offline optimizer baselines · Design-Bench tasks, 100th percentile | normalized performance: 84.85% = 56/66 cases; approximately 2.08%; peak improvement 30% | Mixed |
| Add BOSS sensitivity regularizer to CMA-ES | CMA-ES · Ant Morphology, 100th percentile | normalized performance: 1.915 ± 0.909 -> 2.009 ± 1.540; +9.4% | Helped |
| Add BOSS sensitivity regularizer to BO-qEI | BO-qEI · Superconductor, 100th percentile | normalized performance: 0.394 ± 0.048 -> 0.464 ± 0.013; +7.0% | Helped |
| Add BOSS sensitivity regularizer to ENS-MIN | ENS-MIN · TF-Bind-8, 100th percentile | normalized performance: 0.986 ± 0.006 -> 0.989 ± 0.007; +0.3% | Helped |
| Add BOSS sensitivity regularizer to REINFORCE | REINFORCE · TF-Bind-10, 100th percentile | normalized performance: 0.664 ± 0.061 -> 0.964 ± 0.096; +30.0% | Helped |
| Add BOSS sensitivity regularizer to BO-qEI | BO-qEI · ChEMBL, 100th percentile | normalized performance: 0.659 ± 0.023 -> 0.688 ± 0.000; +2.9% | Helped |
| Use direct Taylor/CDF sensitivity proxy instead of the main BOSS formulation (ablation) | BOSS regularization · Six Design-Bench tasks | normalized performance: 13/18 instances improved; maximum and average improvement 3.1% and 1.25% | Mixed |
| Set sensitivity threshold to an excessively low or high value rather than 0.1 (ablation) | BOSS · Superconductor and TF-Bind-8 | normalized performance | Hurt |
| Increase perturbation sample count beyond 100 (ablation) | BOSS · Superconductor and TF-Bind-8 | performance gain | No clear effect |
| Use perturbation ranges that are excessively low or high instead of the selected ranges (ablation) | BOSS · Superconductor and TF-Bind-8 | normalized performance | Hurt |
| Add BOSS regularization instead of using L1 or L2 regularization (ablation) | Gradient ascent · Superconductor and TF-Bind-8 | objective value | Helped |
| Add BOSS regularizer to surrogate training (ablation) | CBAS and COMs surrogates · TF-Bind-8 and TF-Bind-10, simulated out-of-distribution test data | RMSE | Helped |
Surprising findings
- Plain L1 and L2 regularization do not match the sensitivity regularizer in the reported comparison on Superconductor and TF-Bind-8.
- The 100th-percentile gains are not uniform: at the 75th percentile, several BOSS variants show small decreases (e.g., CbAS on Ant Morphology -0.1%, -0.3%).
Limitations
- Hyperparameters (alpha, omega bounds, lambda) were selected via ablations using the true oracle, although a pseudo-oracle tuning alternative is reported.
- Gains are often small (under 1%) and some configurations decrease performance, especially at the 75th percentile.
- Sensitivity measure is only meaningful within a narrow range of alpha and omega, which requires careful setting.
Experimental setup
Datasets: TF-Bind-8 TF-Bind-10 ChEMBL Ant Morphology D'Kitty Morphology Superconductors
Benchmarks: TF-Bind-8 TF-Bind-10 ChEMBL Ant Morphology D'Kitty Morphology Superconductor
Metrics: normalized oracle performance at 50th, 75th, 100th percentil percentage gain over baseline RMSE on out-of-distribution test data
Baselines: BO-qEI CbAS RoMA ICT CMA-ES COMs MINs REINFORCE GA ENS-MIN ENS-MEAN DDOM
Builds on: Design-Bench CbAS COMs ICT RoMA
Closest papers by meaning
- Learning Surrogates for Offline Black-Box Optimization via Gradient Matching (ICML 2024 · similarity 0.94)
- Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques (NeurIPS 2024 · similarity 0.92)
- Offline Model-Based Optimization via Policy-Guided Gradient Search (AAAI 2024 · similarity 0.91)
- Offline Model-Based Optimization by Learning to Rank (ICLR 2025 · similarity 0.87)
- ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge (NeurIPS 2025 · similarity 0.87)
- Constrained Offline Black-Box Optimization via Risk Evaluation and Management (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.