Research map / Probabilistic and scientific modeling / Bayesian optimization
Multi-Objective Bayesian Optimization with Active Preference Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MBO-APL, a Bayesian optimization method for preference-based multi-objective optimization that interactively learns a Chebyshev-utility preference model from pairwise comparisons and improvement requests, with mutual-information active query selection.
Problem
Finding the single most preferred solution of an expensive multi-objective black-box problem without enumerating the whole Pareto front, when the decision maker's preference is unknown and hard to specify numerically.
Approach
Model the decision maker's utility as a Chebyshev scalarization with a Dirichlet-prior weight vector, estimate it from two weak supervisions (pairwise comparisons and a new improvement-request type giving the dimension to improve most), and run expected improvement jointly over objective GPs and preference-weight posterior, with BALD-style mutual-information queries to reduce interaction cost.
Key results
- On benchmark functions, the proposed method reduces simple regret more efficiently than Random, MOBO-RS and EI-UU and performs similarly to EI with the true preference (EI-TP).
- Active learning drives preference-weight estimation error w_error down to about 0.1 within a few tens of iterations even for L=10.
- In hyper-parameter optimization for cost-sensitive learning and fair classification, the method outperforms MOBO-RS and EI-UU and is comparable to EI-TP, except on CIFAR-10 (L=10) where EI-TP is better.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MI-based active query selection instead of random query selection | MBO-APL preference learning · Preference parameter estimation | w_error | Helped |
| Use MBO-APL instead of Random, MOBO-RS, or EI-UU | Preference-based multi-objective BO · DTLZ1, DTLZ3, Kursawe, and Schaffer2 | simple regret | Helped |
| Estimate preference online with MBO-APL instead of using no preference information | Preference-based multi-objective BO · Cost-sensitive classification: Waveform-5000 and CIFAR-10 | simple regret | Helped |
| Estimate preference online with MBO-APL instead of using no preference information | Preference-based multi-objective BO · Fairness-aware classification: Adult and Bank | simple regret | Helped |
| Use both pairwise comparisons and improvement requests instead of only pairwise comparisons (ablation) | MBO-APL · DTLZ3 | simple regret | Helped |
| Use pairwise comparisons instead of both pairwise comparisons and improvement requests (ablation) | MBO-APL · DTLZ1 and DTLZ3 | simple regret | Hurt |
| Use improvement requests instead of both pairwise comparisons and improvement requests (ablation) | MBO-APL · DTLZ1 and DTLZ3 | simple regret | Hurt |
| Use pairwise comparisons only instead of EI-UU (ablation) | MBO-APL · DTLZ1 and DTLZ3 | simple regret | Helped |
| Use pairwise comparisons only instead of both pairwise comparisons and improvement requests (ablation) | MBO-APL · DTLZ3 | simple regret | Hurt |
| Reduce the number of samples used to approximate expected improvement (ablation) | MBO-APL · Benchmark functions | simple regret | Mixed |
| Replace the CSF utility with a monotonic preferential GP utility (PGPM) (ablation) | MBO-APL · Kursawe and Schaffer2 | simple regret | Helped |
| Replace the CSF utility with a monotonic preferential GP utility (PGPM) (ablation) | MBO-APL · Fonseca-Fleming, Poloni, and Schaffer1 | simple regret | Helped |
Surprising findings
- On CIFAR-10 with 10 objectives, the oracle EI-TP outperforms the proposed method, indicating preference estimation becomes harder in high output dimensions.
Limitations
- Preference is simulated from a sampled true Chebyshev weight vector rather than a real decision maker.
- Preference estimation becomes harder as the number of objectives grows, with a gap to the oracle on CIFAR-10 (L=10).
- The GP-based utility requires more observations to estimate the preference accurately.
Experimental setup
Datasets: DTLZ1 DTLZ3 Kursawe Schaffer2 Fonseca-Fleming Poloni Schaffer1 Waveform-5000 CIFAR-10 Breast Cancer Covertype Adult
Benchmarks: DTLZ1 DTLZ3 Kursawe Schaffer2 Fonseca-Fleming Poloni Schaffer1 Waveform-5000 CIFAR-10 Breast Cancer Covertype Adult
Metrics: simple regret on true utility estimation error of preference weights
Baselines: Random MOBO-RS EI-UU EI-TP
Builds on: Gaussian process regression BALD preferential Gaussian process MOBO-RS EI-UU
Closest papers by meaning
- Preference-Aware Constrained Multi-Objective Bayesian Optimization (Student Abstract) (AAAI 2024 · similarity 0.92)
- User Preference Meets Pareto-Optimality in Multi-Objective Bayesian Optimization (AAAI 2025 · similarity 0.90)
- Bayesian Optimization with Preference Exploration using a Monotonic Neural Network Ensemble (NeurIPS 2025 · similarity 0.89)
- FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning (NeurIPS 2024 · similarity 0.89)
- Principled Preferential Bayesian Optimization (ICML 2024 · similarity 0.88)
- Pareto Front-Diverse Batch Multi-Objective Bayesian Optimization (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.