Research map / Architectures and efficiency / RNNs and normalization
AMLB: an AutoML Benchmark
Pieter Gijsbers, Marcos L. P. Bueno, Stefan Coors, Erin LeDell, Sébastien Poirier, Janek Thomas, Bernd Bischl, Joaquin Vanschoren
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AutoGluon(B) instead of other AutoML frameworks | AutoML frameworks · Binary classification (albert, 1h) | AUC: 0.782 vs 0.737 (autosklearn2) | Improved |
| use AutoGluon stacking ensembles instead of TPOT | TPOT · Binary classification (albert, 1h) | AUC: 0.782 vs 0.718 | Improved |
| tune random forest hyperparameters instead of using an AutoML search (TPOT) | TPOT · All suites, average rank | average rank | Improved |
| use AutoGluon(B) instead of the second best framework | AutoGluon(B) · All suites, CD diagram | average rank | No clear effect |
| increase time budget from 1 hour to 4 hours | AutoML frameworks · All tasks | scaled performance | Improved |
| use AutoGluon(B) stacking ensemble instead of lighter presets | AutoGluon · Inference speed | rows per second | Hurt |
| use cost-frugal optimization that accounts for inference time (FLAML) | FLAML · Inference speed | rows per second | Improved |
| use AutoGluon(B) on datasets with more than 8237 instances | AutoGluon(B) · Bradley-Terry tree, 1h classification | worth parameter | Improved |
| use AutoGluon(B) on small balanced binary tasks | AutoGluon(B) · Small balanced binary classification | worth parameter | Hurt |
| add iterative-learner-only search space with ensembling (MLJAR(B)) bug fixes | MLJAR(B) · Classification and regression suites | errors: 284 | Hurt |
Closest papers by meaning
- AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML (ICML 2025 · similarity 0.84)
- EvoCodeBench: An Evolving Code Generation Benchmark with Domain-Specific Evaluations (NeurIPS 2024 · similarity 0.84)
- Autonomous Evaluation of LLMs for Truth Maintenance and Reasoning Tasks (ICLR 2025 · similarity 0.83)
- EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code (NeurIPS 2025 · similarity 0.83)
- BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.83)
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
