Research map / Learning theory and algorithms / Sample complexity
Pareto-frontier Entropy Search with Variational Lower Bound Maximization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PFEV, a MOBO acquisition function that lower-bounds Pareto-frontier mutual information using a mixture of over- and under-truncated Pareto-frontier normal distributions, with the mixture weight optimized by variational lower bound maximization.
Problem
Multi-objective Bayesian optimization via information gain of the Pareto-frontier requires a predictive distribution truncated by the true Pareto-frontier, which is unobtainable in continuous domains from only a finite subset, so existing methods use only over-truncation and approximation error is u
Approach
Approximate the truncated predictive distribution by a mixture of an over-truncation (PFTN-O) and a new under-truncation (PFTN-U) distribution built from a finite Pareto subset; optimize the mixture weight lambda by maximizing a general variational lower bound of mutual information, which yields a valid MI lower bound and reduces approximation error, plus a beta-prior MAP estimator for variance re
Key results
- PFEV shows superior or comparable RHV to existing methods on GP-derived synthetic functions, with advantages becoming clearer for output dimension L >= 3.
- The MAP-based MC estimator (LB-MAP) improves performance over the naive MC estimator on GP-derived functions with (d,L)=(2,6) and (3,6).
- Optimized lambda takes intermediate values in (0,1], indicating both over- and under-truncated distributions are mixed during BO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PFEV instead of existing MOBO methods | MOBO on GP-generated synthetic functions · GP-generated synthetic functions | RHV | Mixed |
| Use PFEV instead of the compared MOBO methods | MOBO on benchmark functions · Fonseca-Fleming, Kursawe, Viennet, FES3, Fonseca+Viennet, and FES3+Kursawe | RHV | Helped |
| Use PFEV instead of the compared MOBO methods | LightGBM class-weight optimization · Abalone, Waveform, Pageblocks, and Gesturephase | hyper-volume of test accuracies | Mixed |
| Use the MAP-based lower-bound estimator instead of the naive MC estimator (ablation) | PFEV · GP-generated synthetic functions, (d, L) = (2, 6) and (3, 6) | RHV | Helped |
| Use the MAP-based MC estimator instead of naive MC (ablation) | PFEV lower-bound estimation · GP-generated two-objective problem, d = 1 | MSE | Helped |
| Use MAP with r = 1 rather than MAP with r = √(10/K) (ablation) | PFEV lower-bound estimation · GP-generated two-objective problem, d = 1 | MSE | Hurt |
| Add conditioning on the optimal input and Pareto frontier (ablation) | PFEV joint-entropy-search extension · MOBO | performance | No clear effect |
Surprising findings
- The theoretical guarantee of MI approximation and convergence is not established, as acknowledged by the authors.
Limitations
- No theoretical guarantee of MI approximation accuracy or BO convergence.
- Mixture uses only one parameter lambda because only K (about 10) samples are available; a more flexible variational distribution may be needed.
- Acquisition maximization uses DIRECT, and gradient-based optimization is only discussed, not fully evaluated.
Experimental setup
Datasets: GP-generated synthetic functions Fonseca-Fleming Kursawe Viennet FES3 Abalone Waveform Pageblocks Gesturephase
Benchmarks: GP-derived synthetic functions Fonseca-Fleming Kursawe Viennet FES3 LightGBM class-weight optimization
Metrics: relative hyper-volume (RHV)
Baselines: EHVI PFES {PF}^2ES JES random search ParEGO MOBO-RS MESMO
Builds on: Pareto-frontier entropy search max-value entropy search PFTN {PF}^2ES
Closest papers by meaning
- Multi-objective Bayesian Optimization using Pareto-frontier Entropy (ICML 2020 · similarity 0.91)
- Joint Entropy Search for Multi-Objective Bayesian Optimization (NeurIPS 2022 · similarity 0.88)
- Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models (AAAI 2025 · similarity 0.86)
- BOtied: Multi-objective Bayesian optimization with tied multivariate ranks (ICML 2024 · similarity 0.86)
- MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions (NeurIPS 2025 · similarity 0.85)
- User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimization (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.