Research map / Probabilistic and scientific modeling / Bayesian optimization
Warm Starting CMA-ES for Hyperparameter Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize CMA-ES from the promising-region distribution estimated from source-task observations | CMA-ES · HPO tasks using a dataset subset as source | best objective value | Helped |
| Initialize CMA-ES from a promising-region distribution estimated on another dataset | CMA-ES · Fashion-MNIST from MNIST; CIFAR-10 from SVHN | best objective value: approximately 25 and 30 evaluations | Helped |
| Use source-task information with higher subset ratio (ablation) | WS-CMA-ES · MNIST and Fashion-MNIST HPO | convergence speed | Helped |
| Vary the source-task offset away from the target offset (ablation) | WS-CMA-ES · Sphere and Rotated Ellipsoid Functions | best evaluation value f | Mixed |
| Replace source-based initialization with reuse of the source-task final Gaussian distribution (ablation) | CMA-ES · Sphere and Rotated Ellipsoid Functions | best evaluation value f | Mixed |
| Keep sampling from the source-task GMM without adapting during target optimization (ablation) | WS-CMA-ES · Sphere and Rotated Ellipsoid Functions | best evaluation value f | Hurt |
| Restrict the warm-started covariance matrix to a diagonal (ablation) | WS-CMA-ES · HPO under a severely limited budget | best objective value | No clear effect |
| Vary alpha from 0.05 to 0.25 (ablation) | WS-CMA-ES · MLP on MNIST with subset source | best objective value | No clear effect |
| Increase gamma across the tested range (ablation) | WS-CMA-ES · MLP on MNIST with subset source | best objective value | Hurt |
| Use alpha=0.05 on a dissimilar source-target task (ablation) | WS-CMA-ES · Rotated Ellipsoid Function, offset b=0.2 | best evaluation value f | Hurt |
| Use gamma=0.05 on a dissimilar source-target task (ablation) | WS-CMA-ES · Rotated Ellipsoid Function, offset b=0.2 | best evaluation value f | Hurt |
Closest papers by meaning
- Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning (AAAI 2024 · similarity 0.86)
- BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems (NeurIPS 2026 · similarity 0.85)
- UQ-Guided Hyperparameter Optimization for Iterative Learners (NeurIPS 2024 · similarity 0.85)
- Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning (NeurIPS 2025 · similarity 0.84)
- Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.84)
- Efficient Hyperparameter Optimization with Adaptive Fidelity Identification (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.