Research map / Probabilistic and scientific modeling / Bayesian optimization
BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BOFormer, a Transformer-based generalized DQN that learns a non-Markovian acquisition function for multi-objective Bayesian optimization, trained on synthetic GP functions and transferred zero-shot to synthetic and 3DGS hyperparameter tasks.
Problem
Learned acquisition functions for multi-objective Bayesian optimization suffer from the hypervolume identifiability issue because hypervolume improvement is history-dependent (non-Markovian), so single-step state representations cannot distinguish scenarios.
Approach
Generalized DQN for non-Markovian environments implemented with a Transformer over history, using a domain-agnostic Q-augmented per-candidate observation representation, normalized hypervolume-improvement reward, prioritized trajectory replay, and demo-policy-guided exploration.
Key results
- On synthetic functions, BOFormer achieves the best or within 1% of the best hypervolume on several tasks (e.g., 0.7377 on ARa vs 0.6290-0.7122 for rule-based baselines at step 100), but not on BC.
- On 3DGS hyperparameter optimization, BOFormer is best or near-best on all five scenes (e.g., 0.9508 on Lego and 0.9745 on Ship at step 30).
- Markovian (w=1) BOFormer underperforms non-Markovian variants, supporting that sequence modeling mitigates hypervolume identifiability.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BOFormer instead of the strongest listed baseline | MOBO optimization · AR, ARa, RBF, Chairs, Lego, Materials, Ship | hypervolume: 0.5787 -> 0.5900; 0.7122 -> 0.7377; 0.8731 -> 0.8751; 0.9159 -> 0.9162; 0.9409 -> 0.9508; 0.9217 -> 0.9224; 0.9661 -> 0. | Helped |
| Use BOFormer instead of the strongest listed baseline | MOBO optimization · BC, DRa, Matern, BCD, Mic | hypervolume: 0.4773 -> 0.4476; 0.9573 -> 0.9461; 0.8696 -> 0.8642; 0.3697 -> 0.3617; 0.8871 -> 0.8816 | Hurt |
| Add Q-augmented observation representation (ablation) | BOFormer, compared with QT without Q-augmented representation · Synthetic functions | hypervolume | Mixed |
| Use non-Markovian sequence length (w > 1) rather than Markovian sequence length (w = 1) (ablation) | BOFormer · MOBO benchmark tasks | hypervolume | Helped |
| Remove temporal information from the representation (ablation) | BOFormer · Training | training loss stability | Hurt |
| Remove both temporal information and positional encoding (ablation) | BOFormer · Training | training loss stability | Hurt |
Surprising findings
- Plain Q-Transformer without Q-augmented representation performs poorly, indicating sequence modeling alone does not guarantee efficient Pareto search.
- The direct implementation with full posterior over all domain points reportedly needs about 75 days of training for 1000 iterations on a 50-point domain.
Limitations
- Performance is not as strong on Branin and Currin functions, attributed to larger lengthscales.
- Continuous high-dimensional domains rely on Sobol-grid approximate maximization, which may be limiting in high dimensions.
- Direct full-domain implementation is computationally infeasible, motivating the domain-agnostic design.
Experimental setup
Datasets: synthetic GP functions HPO-3DGS
Benchmarks: Ackley-Rastrigin Ackley-Rosenbrock Branin-Currin Dixon-Rastrigin RBF Matern Branin-Currin-Dixon HPO-3DGS
Metrics: attained hypervolume simple regret performance profiles
Baselines: qNEHVI qParEGO NSGA-II qHVKG JES FSAF Decision Transformer Q-Transformer OptFormer
Builds on: DQN DRQN FSAF Decision Transformer Q-Transformer
Closest papers by meaning
- Large Language Models to Enhance Bayesian Optimization (ICLR 2024 · similarity 0.89)
- MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions (NeurIPS 2025 · similarity 0.87)
- Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization (ICML 2025 · similarity 0.87)
- PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance (AAAI 2024 · similarity 0.87)
- Latent Bayesian Optimization via Autoregressive Normalizing Flows (ICLR 2025 · similarity 0.87)
- Simple Extensions of Single-Objective Acquisition Functions and Hedge Strategies for Multi-Objective Bayesian Optimization (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.