Research map / Probabilistic and scientific modeling / Bayesian optimization
Optimization Proxies using Limited Labeled Data and Training Time -- A Semi-Supervised Bayesian Neural Network Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a semi-supervised "Sandwich" Bayesian neural network optimization proxy that alternates supervised cost learning with feasibility-based unsupervised training on augmented inputs, and derives tighter Bernstein-based error bounds using mean predictive variance from the BNN.
Problem
Learning optimization proxies for constrained, non-convex optimization (ACOPF) when labeled data and training time are both severely limited, and providing tight out-of-sample error bounds for them.
Approach
Train a BNN with alternating supervised (labeled cost-optimal) and unsupervised (feasibility-loss on unlabeled inputs with Gaussian likelihood) SVI steps; select posterior weight samples that minimize max equality gap (SvP); and use 2×mean predictive variance as a proxy for total variance in error inside a theoretical Bernstein bound for probabilistic confidence bounds.
Key results
- On case57, Sandwich BNN SvP reaches Max Eq. 0.027 vs 0.221 for best DNN baseline (LD+MAE), with Gap% 0.928 vs 3.709.
- On case118, Sandwich BNN SvP achieves Max Eq. 0.089 vs 1.284 for LD+MAE, and Max Ineq. comparable while Gap% is 1.484.
- Bernstein bound with 2×MPV gives 0.010 pu error bound for case118 voltage vs 0.064 pu for Hoeffding and 0.018 pu for empirical Bernstein.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Sandwich BNN SvP instead of the LD + MAE baseline | ACOPF proxy · ACOPF case57 | Max Eq.: 0.221 -> 0.027 | Helped |
| Use Sandwich BNN SvP instead of the LD + MAE baseline | ACOPF proxy · ACOPF case118 | Max Eq.: 1.284 -> 0.089 | Helped |
| Use Sandwich BNN SvP instead of the LD + MAE baseline | ACOPF proxy · ACOPF case500 | Max Eq.: 11.166 -> 0.770 | Helped |
| Use Sandwich BNN SvP instead of the LD + MAE baseline | ACOPF proxy · ACOPF case2000 | Max Eq.: 18.870 -> 5.114 | Helped |
| Add Selection via Posterior (ablation) | Sandwich BNN · ACOPF case57 | Max Eq.: 0.045 -> 0.027 | Helped |
| Add sandwich semi-supervised training (ablation) | Supervised BNN SvP · ACOPF case57 | Max Eq.: 0.083 -> 0.027 | Helped |
| Replace BNN with DNN (ablation) | Sandwich BNN · ACOPF case118 | Max Eq.: 0.100 -> 0.585 | Hurt |
| Add sandwich semi-supervised training (ablation) | Supervised BNN SvP · ACOPF case500 | Max Eq.: 2.204 -> 0.770 | Helped |
| Use Sandwich BNN instead of Supervised BNN (ablation) | BNN proxy · Non-OPF (70,20,50) | Max-Eq violation: 0.484 -> 0.298 | Helped |
| Use Sandwich BNN instead of Supervised BNN (ablation) | BNN proxy · Non-OPF (70,20,50) | Gap%: 2.6 -> 7.1 | Hurt |
| Use Sandwich BNN instead of Supervised BNN (ablation) | BNN proxy · Non-OPF (20,10,20) | Max-InEq violation: 0.108 -> 0.000 | Helped |
| Use the theoretical Bernstein bound with 2 × MPV instead of Hoeffding's bound | BNN probabilistic confidence bound · ACOPF case118 voltage error | error bound (pu): 0.064 -> 0.010 | Helped |
Surprising findings
- On case2000, the Sandwich DNN baseline performs very poorly (Gap% 54.5), while the BNN variants keep gaps near 0.5%.
- The supervised BNN generally has lower variability across trials than the sandwich BNN, and the sandwich method can yield slightly higher inequality gaps on larger cases.
Limitations
- Max Eq. values remain high on case500 and case2000 (above 0.7 and 5 respectively) within 600 s of training, so raw predictions are not feasible without projection.
- Bernstein bound relies on an empirical hypothesis that 2×MPV upper-bounds total variance in error, validated only empirically with alpha=2 on tested cases.
- Max inequality violations are not suppressed without a bound repair layer, which the BNN does not use.
Experimental setup
Datasets: OPFDataset (Torch_Geometric)
Benchmarks: OPFDataset case57 OPFDataset case118 OPFDataset case500 OPFDataset case2000 Donti et al. non-convex QP-type problem
Metrics: Gap% Max Eq. Mean Eq. Max Ineq. Mean Ineq. Bernstein error bound
Baselines: Naïve MAE Naïve MSE MAE + Penalty MSE + Penalty LD + MAE Sandwich DNN Supervised BNN
Builds on: ML4OPF Donti et al. DC3 / feasibility-based learning Hoeffding's inequality empirical Bernstein bound stochastic variational inference
Closest papers by meaning
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.88)
- Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints (ICML 2024 · similarity 0.85)
- FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees (NeurIPS 2025 · similarity 0.85)
- Bayesian Optimization via Continual Variational Last Layer Training (ICLR 2025 · similarity 0.85)
- Bayesian Optimization of Function Networks with Partial Evaluations (ICML 2024 · similarity 0.84)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.