Research map / Probabilistic and scientific modeling / Bayesian optimization
PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PG-LBO, which adds pseudo-label weighted retraining on unlabeled VAE latent samples and a GP-guidance deep-kernel loss to LBO for better latent spaces in VAE-based high-dimensional Bayesian optimization.
Problem
VAE-based Bayesian optimization for high-dimensional structured inputs under-uses unlabeled data and handles labeled data with complex or costly add-ons, so the latent space is not discriminative enough for BO.
Approach
Use GP posterior means as pseudo-labels on unlabeled latent samples to rank them and weight VAE retraining, filter them by GP predictive variance with a dynamic EMA threshold, and train the VAE encoder jointly with a frozen GP via a deep-kernel-style MSE plus variance loss.
Key results
- On topology, PG-LBO reaches 0.866±0.0070 cosine similarity, versus 0.855 for LBO and 0.859 for LOL-BO.
- On molecule PlogP, PG-LBO reaches 31.77±0.3287, beating LOL-BO (27.53±2.3927) and LBO (7.01±0.5174).
- On expression, PG-TLBO reaches 0.175±0.1443, the best value in the table, while PG-LBO reaches 0.227±0.1864 versus LBO's 0.314.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pseudo-label training and GP guidance to LBO | LBO · Topology shape fitting | cosine similarity: 0.855$\pm$0.0072 -> 0.866$\pm$0.0070 | Helped |
| Add pseudo-label training and GP guidance to LBO | LBO · Expression reconstruction | distance: 0.314$\pm$0.1436 -> 0.227$\pm$0.1864 | Helped |
| Add pseudo-label training and GP guidance to LBO | LBO · Chemical design | PlogP: 7.01$\pm$0.5174 -> 31.77$\pm$0.3287 | Helped |
| Add pseudo-label training (ablation) | PG-LBO · Topology shape fitting | cosine similarity | Helped |
| Add GP guidance (ablation) | PG-LBO · Topology shape fitting | cosine similarity | Helped |
| Use random sampling instead of noisy sampling for pseudo-label data (ablation) | PG-LBO · Topology shape fitting | cosine similarity | Hurt |
| Use heuristic sampling instead of noisy sampling for pseudo-label data (ablation) | PG-LBO · Topology shape fitting | cosine similarity | No clear effect |
| Filter pseudo-label data using an uncertainty threshold (ablation) | PG-LBO · Topology shape fitting | cosine similarity | Helped |
| Use a dynamic uncertainty threshold instead of a fixed threshold (ablation) | PG-LBO · Topology shape fitting | cosine similarity | Helped |
| Vary λG among the tested settings (ablation) | PG-LBO · Topology shape fitting, expression reconstruction, and chemical design [1000] | final optimization score | No clear effect |
| Use linearly increasing λP instead of fixed λP (ablation) | PG-LBO · Topology shape fitting, expression reconstruction, and chemical design [1000] | final optimization score | Helped |
| Apply PG-LBO components to T-LBO | T-LBO · Topology shape fitting, expression reconstruction, and chemical design | optimization score | Helped |
Surprising findings
- Random unlabeled sampling for pseudo-label training performs even worse than the LBO baseline, showing pseudo-label training can hurt when sampled points are poor.
- The authors' reproduced T-LBO results deviated significantly from those reported in the original T-LBO paper.
Limitations
- Pseudo-labels with large errors can still bias the model, so the method depends on sampling quality and filtering.
- Gains over LBO are small on the topology task, and results vary strongly across tasks and seeds (high std in expression).
- Experiments are limited to three tasks with 5 seeds, and T-LBO results could not be reproduced as reported.
Experimental setup
Datasets: topology shape dataset (10000 images) expression reconstruction data (40,000 points) ZINC250K
Benchmarks: topology shape fitting expression reconstruction chemical design (PlogP on ZINC250K)
Metrics: best objective value over 500 evaluations mean and standard deviation over 5 seeds
Baselines: LSBO LBO T-LBO LOL-BO TSBO
Builds on: LBO T-LBO LSBO FreeMatch Meta Pseudo Labels
Closest papers by meaning
- Latent Bayesian Optimization via Autoregressive Normalizing Flows (ICLR 2025 · similarity 0.88)
- Bayesian Basis Function Approximation for Scalable Gaussian Process Priors in Deep Generative Models (ICML 2025 · similarity 0.88)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.87)
- High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling (ICML 2024 · similarity 0.87)
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.87)
- Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.