Research map / Probabilistic and scientific modeling / Bayesian optimization
MALIBO: Meta-learning for Likelihood-free Bayesian Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MALIBO, a meta-learning likelihood-free BO method that meta-learns a neural classifier over acquisition utilities, adapts via Laplace-approximated Bayesian task embeddings with Thompson sampling, and adds gradient boosting as a residual model.
Problem
Meta-learning Bayesian optimization is hampered by GP surrogates that scale poorly and are sensitive to heterogeneous observation scales, noise types, and task uncertainty during warm-starting of new tasks.
Approach
Bypass the surrogate model by meta-learning a likelihood-free classifier (LFBO-style utility-weighted classification) with a shared feature network and regularized Gaussian task embeddings; adapt to new tasks via a Laplace-approximated posterior over the embedding, Thompson sampling for exploration, and gradient boosting as a residual correction.
Key results
- MALIBO consistently achieves strong anytime performance across NASBench201, HPOBench and HPO-B, outperforming other meta-learning BO methods; it outperforms other methods by a large margin on HPOBench.
- The Laplace-approximated MALIBO takes about 0.1 s per iteration on NASBench201 versus about 10 s for SVI and 100 s for HMC variants, with comparable regret.
- On multiplicative-noise Hartmann3D, GP-based methods such as RGPE degrade significantly with increasing noise while MALIBO remains robust.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Thompson sampling for exploration (ablation) | MALIBO · Forrester and quadratic synthetic functions | optimization performance | Helped |
| add gradient boosting as a residual prediction model (ablation) | MALIBO · HPOBench and NASBench201 | normalized regret | Helped |
| add a mean prediction layer (ablation) | MALIBO · real-world AutoML benchmarks | warm-starting performance | Helped |
| remove the task prediction layer and use only the task-agnostic component (ablation) | MALIBO · NASBench201 | normalized regret | Helped |
| remove the task prediction layer and use only the task-agnostic component (ablation) | MALIBO · HPOBench and HPO-B | normalized regret | Hurt |
| replace gradient boosting with a random forest classifier (ablation) | MALIBO · real-world AutoML benchmarks | normalized regret | Hurt |
| replace gradient boosting with an MLP classifier (ablation) | MALIBO · real-world AutoML benchmarks | normalized regret | Hurt |
| use probit approximation for the marginalized acquisition function and omit gradient boosting (ablation) | MALIBO · Forrester and quadratic synthetic functions | optimization performance | Hurt |
| use Thompson sampling without gradient boosting (ablation) | MALIBO · HPOBench and NASBench201 | normalized regret | Hurt |
| remove the mean prediction layer (ablation) | MALIBO · real-world AutoML benchmarks | warm-starting performance | Hurt |
| use only the task-agnostic mean prediction layer, without the task prediction layer (ablation) | MALIBO · HPOBench and HPO-B | normalized regret | Hurt |
| replace Laplace approximation with stochastic variational inference (ablation) | MALIBO · NASBench201 | runtime per iteration (seconds): 0.1 vs around 10 seconds | Hurt |
Surprising findings
- MALIBO without gradient boosting or with only Thompson sampling underperforms when meta-data is limited, and the MEAN-only variant is best on NASBench201 but worse elsewhere.
- MetaBO, a meta-learned method, fails to transfer knowledge on NASBench201 due to task diversity.
Limitations
- Exploration/exploitation threshold tau (gamma=1/3) is set heuristically rather than treated probabilistically.
- Regularization hyperparameter lambda may be suboptimal in other scenarios.
- Uni-modal Gaussian task prior may be restrictive for complex task distributions; Gaussian mixture extension is unverified.
Experimental setup
Datasets: NASBench201 HPOBench HPO-B Hartmann3D function ensemble Forrester functions quadratic functions
Benchmarks: NASBench201 HPOBench HPO-B Hartmann3D synthetic ensemble
Metrics: normalized regret immediate regret average rank runtime
Baselines: Random search LFBO GP ABLR BANNER RGPE GC3P FSBO MetaBO PFN DRE LFBO+BB
Builds on: LFBO BORE TPE RGPE BANNER
Closest papers by meaning
- MetaGameBO: Hierarchical Game-Theoretic Driven Robust Meta-Learning for Bayesian Optimization (AAAI 2026 · similarity 0.91)
- SMOG: Scalable Meta-Learning for Multi-Objective Bayesian Optimization (NeurIPS 2026 · similarity 0.90)
- Meta-Learning Reliable Priors in the Function Space (NeurIPS 2021 · similarity 0.89)
- End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes (NeurIPS 2023 · similarity 0.89)
- Large Language Models to Enhance Bayesian Optimization (ICLR 2024 · similarity 0.88)
- More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.