Research map / Probabilistic and scientific modeling / Bayesian optimization
An Empirical Bayes Approach to Optimizing Machine Learning Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace BayesOpt hyperparameter selection with EB-Hyp averaging, no validation set | LDA · 20 Newsgroups | predictive log likelihood: -361661 -> -351911 | Helped |
| use EB-Hyp instead of BayesOpt when a validation set is available | LDA · 20 Newsgroups | predictive log likelihood: -357648 -> -357650 | No clear effect |
| remove the separate validation set from BayesOpt (ablation) | BayesOpt · LDA 20 Newsgroups | predictive log likelihood: -357648 -> -361661 | Hurt |
| remove the separate validation set from EB-Hyp (ablation) | EB-Hyp · LDA 20 Newsgroups | predictive log likelihood: -357650 -> -351911 | Helped |
| replace BayesOpt with EB-Hyp, both without validation | DLGM · Labeled Faces in the Wild | predictive log likelihood: -15970 -> -15872 | Helped |
| replace BayesOpt with EB-Hyp, both with validation | DLGM · Labeled Faces in the Wild | predictive log likelihood: -17071 -> -16375 | Helped |
| remove the validation set from BayesOpt (ablation) | BayesOpt · DLGM Labeled Faces in the Wild | predictive log likelihood: -17071 -> -15970 | Helped |
| remove the validation set from EB-Hyp (ablation) | EB-Hyp · DLGM Labeled Faces in the Wild | predictive log likelihood: -16375 -> -15872 | Helped |
| random hyperparameter search instead of BayesOpt | LDA · 20 Newsgroups | predictive log likelihood: -357648 -> -2666074 | Hurt |
| random hyperparameter search instead of BayesOpt | DLGM · Labeled Faces in the Wild | predictive log likelihood: -17071 -> -17271 | Hurt |
Closest papers by meaning
- Bayesian Optimization Meets Bayesian Optimal Stopping (ICML 2019 · similarity 0.88)
- Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning (NeurIPS 2019 · similarity 0.87)
- Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.87)
- Bayesian Optimization with Robust Bayesian Neural Networks (NeurIPS 2016 · similarity 0.87)
- Hypervolume Knowledge Gradient: A Lookahead Approach for Multi-Objective Bayesian Optimization with Partial Information (ICML 2023 · similarity 0.87)
- Efficient Hyperparameter Optimization with Adaptive Fidelity Identification (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.