Research map / Learning theory and algorithms / Prediction and decision losses
Benchopt: Reproducible, efficient and collaborative optimization benchmarks
Thomas Moreau, Mathurin Massias, Alexandre Gramfort, Pierre Ablin, Pierre-Antoine Bannier, Benjamin Charlier, Mathieu Dagréou, Tom Dupre la Tour, Ghislain Durif, Cássio Fraga Dantas, Quentin Klopfenstein, Johan Larsson and 9 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use appropriate data augmentation (ablation) | ResNet18 · CIFAR-10 | test error %: about 18 -> about 8 | Improved |
| combine weight decay with proper LR schedule and momentum (ablation) | ResNet18 SGD · CIFAR-10 | test error % | Improved |
| add Lookahead on top of best SGD with cosine LR (ablation) | SGD · CIFAR-10 | convergence speed | Improved |
| use GPU solvers (cuML, snapML) | logistic regression solvers · ijcnn1 | time to suboptimality | Improved |
| scale features to unit std | L2 logistic regression solvers · madelon | convergence time: condition number divided by 5.9 | Improved |
| scale features to unit std | SAG/SAGA · news20.binary | convergence time: condition number multiplied by 407 | Hurt |
| use stochastic variance-reduced solvers instead of quasi-Newton | SAG/SAGA/SVRG · criteo | convergence time | Improved |
| use train-validation-test split instead of reporting best test error | ResNet18 ablation · CIFAR-10 | ablation conclusions | No clear effect |
| use working-set strategies with coordinate descent instead of full gradient ISTA/FISTA | Lasso solvers · Lasso benchmarks | time to suboptimality | Improved |
Closest papers by meaning
- DeepOBS: A Deep Learning Optimizer Benchmark Suite (ICLR 2019 · similarity 0.88)
- Efficient Lifelong Model Evaluation in an Era of Rapid Progress (NeurIPS 2024 · similarity 0.87)
- BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems (NeurIPS 2026 · similarity 0.87)
- Efficient Distributed Optimization under Heavy-Tailed Noise (ICML 2025 · similarity 0.86)
- AutoOpt: A Dataset and a Unified Framework for Automating Optimization Problem Solving (NeurIPS 2025 · similarity 0.86)
- Benchmark Data Repositories for Better Benchmarking (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
