Research map / Label-efficient and robust learning / Semi-supervised learning
High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TSBO, a teacher-student semi-supervised framework for latent-space Bayesian optimization that uses optimized unlabeled data samplers (GEV-based and learned Gaussian) and uncertainty-aware pseudo labels to improve GP surrogate generalization under tight labeled budgets.
Problem
Latent space Bayesian optimization in high-dimensional spaces is sample-inefficient because the GP surrogate overfits when labeled queries are scarce and expensive.
Approach
Integrates a teacher-student semi-supervised paradigm into BO: a student GP trained on sampled unlabeled locations with teacher pseudo labels gives MSE feedback on labeled data to regularize the teacher; the unlabeled samples come from an optimized sampler (EVT/GEV or a parameterized distribution trained via reparameterization on student feedback), and teacher uncertainty is propagated into the st
Key results
- On Penalized LogP after 250 queries, TSBO-Gaussian reaches 25.02±4.79 versus 5.70±1.25 for T-LBO.
- On the expression task, TSBO-Gaussian reaches 0.240±0.168 (lower is better) versus 0.572±0.268 for T-LBO.
- The paper claims up to 364.2x reduction in labeled data use relative to baselines in the broader comparison.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Gaussian rather than GEV unlabeled-data sampling | TSBO · Expression, Penalized LogP, and Ranolazine MPO after 250 queries | best value: 0.396±0.070 -> 0.240±0.168; 18.40±7.890 -> 25.02±4.794; 0.708±0.032 -> 0.744±0.030 | Mixed |
| Add optimized unlabeled-data sampling instead of random sampling (ablation) | TSBO · Penalized LogP | best value: 4.881±1.416 -> 25.020±4.794 | Helped |
| Add uncertainty awareness to the teacher-student model (ablation) | TSBO · Penalized LogP | best value: 12.568±7.965 -> 25.020±4.794 | Helped |
| Add student feedback to the teacher (ablation) | TSBO · Penalized LogP | best value: 17.557±6.998 -> 25.020±4.794 | Helped |
| Set feedback weight λ to 0.001 rather than 0.1 (ablation) | TSBO-Gaussian · Penalized LogP | best value: 25.021±4.794 vs 21.106±8.960 | No clear effect |
| Set feedback weight λ to 0.01 rather than 0.1 (ablation) | TSBO-Gaussian · Penalized LogP | best value: 25.021±4.794 vs 21.384±1.533 | No clear effect |
| Set feedback weight λ to 1 rather than 0.1 (ablation) | TSBO-Gaussian · Penalized LogP | best value: 25.021±4.794 vs 21.122±7.494 | No clear effect |
| Set feedback weight λ to 0.001 rather than 0.1 (ablation) | TSBO-Gaussian · Expression | best value: 0.450±0.130 vs 0.240±0.168 | No clear effect |
| Set feedback weight λ to 0.01 rather than 0.1 (ablation) | TSBO-Gaussian · Expression | best value: 0.450±0.130 vs 0.433±0.260 | No clear effect |
| Set feedback weight λ to 1 rather than 0.1 (ablation) | TSBO-Gaussian · Expression | best value: 0.450±0.130 vs 0.474±0.113 | No clear effect |
| Choose the 30 highest-label examples for validation rather than 30 random examples (ablation) | TSBO-Gaussian · Penalized LogP | best value: 21.60 -> 25.02 | Helped |
| Choose the 30 highest-label examples for validation rather than all labeled data (ablation) | TSBO-Gaussian · Penalized LogP | best value: 22.65 -> 25.02 | Helped |
Surprising findings
- A random unlabeled sampler performs far worse than the optimized samplers (4.881 vs 25.020 on Penalized LogP), and removing student feedback drops performance to 17.557.
Limitations
- Uncertainty quantification is heuristic (MLP with diagonal covariance); authors suggest deep ensembles or Bayesian neural networks as future work.
- Built on T-LBO and evaluated only on two VAE-based latent-space tasks with a small number of runs (5).
Experimental setup
Datasets: 40,000 single-variable arithmetic expressions ZINC250K
Benchmarks: arithmetic expression reconstruction penalized LogP chemical design Ranolazine MPO chemical design
Metrics: best value found mean and standard deviation over runs negative log-likelihood of GP posterior
Baselines: Sobol LS-BO W-LBO T-LBO LOL-BO PG-LBO MolDQN
Builds on: T-LBO latent space bayesian optimization gaussian process reparameterization trick extreme value theory
Closest papers by meaning
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.88)
- PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance (AAAI 2024 · similarity 0.87)
- Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization (ICML 2025 · similarity 0.86)
- Vanilla Bayesian Optimization Performs Great in High Dimensions (ICML 2024 · similarity 0.86)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.86)
- BOIDS: High-Dimensional Bayesian Optimization via Incumbent-Guided Direction Lines and Subspace Embeddings (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.