Research map / Learning theory and algorithms / Prediction and decision losses
Weighted Sampling for Combined Model Selection and Hyperparameter Tuning
Dimitrios Sarigiannis, Thomas Parnell, Haralampos Pozidis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add weighted model sampling to SH0 | SH0 (exploitative) · 67 OpenML datasets | average rank (validation/test loss): p = 3.9e-8 (validation) | Improved |
| Add weighted model sampling to SH1 | SH1 · 67 OpenML datasets | average rank: p = 1.3e-7 (validation) | Improved |
| Add weighted model sampling to SH2 | SH2 (explorative) · 67 OpenML datasets | average rank: p = 2.4e-6 (test) | Improved |
| Use more explorative SH schedule | Successive Halving · 67 OpenML datasets | average rank | Improved |
| Add weighted sampling to Hyperband | Hyperband · 67 OpenML datasets | average rank: p = 8.3e-7 (validation) | Improved |
| Add weighted sampling to random search | Random search · 67 OpenML datasets | average rank: p = 8.3e-7 (validation) | Improved |
| Add weighted sampling to SH2 in RS/HB/SH comparison | Successive Halving · 67 OpenML datasets | average rank: p = 7.3e-6 (validation) | Improved |
| Compare Hyperband with weighted sampling vs SH with weighted sampling | Hyperband · 67 OpenML datasets | validation loss: p = 3.4e-1 | No clear effect |
| Compare Hyperband with weighted sampling vs SH with weighted sampling | Hyperband · 67 OpenML datasets | test loss: p = 1.9e-1 | No clear effect |
| Compare Hyperband vs random search without weighting | Hyperband · 67 OpenML datasets | test loss: p = 9.8e-2 | No clear effect |
| Compare Hyperband vs random search without weighting | Hyperband · 67 OpenML datasets | validation loss: p = 9.2e-5 | Improved |
| Explorative SH versus Hyperband and RS, no weighting | Explorative SH · 67 OpenML datasets | average rank: p = 3.7e-4 (HB vs SH, validation) | Improved |
Closest papers by meaning
- HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML (NeurIPS 2021 · similarity 0.87)
- Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function (NeurIPS 2025 · similarity 0.86)
- Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyond (ICML 2024 · similarity 0.86)
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.86)
- Cost-Sensitive Freeze-thaw Bayesian Optimization for Efficient Hyperparameter Tuning (NeurIPS 2025 · similarity 0.86)
- TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
