Research map / Probabilistic and scientific modeling / Molecules and proteins
Personalized Benchmarking with the Ludwig Benchmarking Toolkit
Avanika Narayan, Piero Molino, Karan Goel, Willie Neiswanger, Christopher Re
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DistilBERT rather than other evaluated models | Text classification models · Text classification across 9 datasets | inference efficiency-performance tradeoff | Improved |
| Use BERT rather than RoBERTa or T5-small | Text classification models · Text classification across datasets | inference latency | Improved |
| Increase model size / use larger models | Text classification models · Text classification across 9 datasets | accuracy | Improved |
| Use pretrained models rather than models trained from scratch | Text classification models · Text classification datasets | accuracy variance | Mixed |
| Evaluate on midsize rather than small datasets | Models compared across dataset sizes · Text classification datasets | variance in model performance | Improved |
| Increase average sentence length | Text classification datasets · Text classification datasets | accuracy | Improved |
| Fine-tune BERT | BERT models · TextAttack adversarial attacks across datasets | attack success rate | Hurt |
| Use RoBERTa instead of BERT | Pretrained text classification models · TextAttack adversarial attacks across datasets | attack success rate | Improved |
| Use RNN instead of BERT | Text classification models · TextAttack adversarial attacks across datasets | attack success rate | Improved |
| Use pretrained rather than scratch-trained models | Text classification models · Model convergence | epochs until best checkpoint | Mixed |
Closest papers by meaning
- How Benchmark Prediction from Fewer Data Misses the Mark (NeurIPS 2025 · similarity 0.89)
- Benchmark Data Repositories for Better Benchmarking (NeurIPS 2024 · similarity 0.88)
- From Crowdsourced Data to High-quality Benchmarks: Arena-Hard and Benchbuilder Pipeline (ICML 2025 · similarity 0.86)
- Fine-Grained Benchmark Generation for Comprehensive Evaluation of Foundation Models (NeurIPS 2026 · similarity 0.86)
- Efficient Lifelong Model Evaluation in an Era of Rapid Progress (NeurIPS 2024 · similarity 0.86)
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
