Research map / Optimization / Optimizers
Better Parameter-Free Stochastic Optimization with ODE Updates for Coin-Betting
Keyi Chen, John Langford, Francesco Orabona
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CODE instead of Coin | Parameter-free coin-betting optimization · 17 regression datasets, best fixed learning rate | mean normalized absolute loss: CODE wins Coin by ~0.1984 | Improved |
| Use CODE instead of AdaGrad | Optimization on 17 regression datasets · 17 regression datasets, best fixed learning rate | mean normalized absolute loss: 1.1227 -> 0.7825 | Improved |
| Use CODE instead of AdaGrad | Optimization on 21 classification datasets · 21 classification datasets, best fixed learning rate | mean normalized 0-1 loss: 0.1778 -> 0.1872 | Hurt |
| Use CODE instead of AdaGrad | Optimization on 17 regression datasets · 17 regression datasets, tuned learning rates | mean normalized absolute loss: 0.7085 -> 0.7825 | Hurt |
| Use CODE instead of AdaGrad | Optimization on 21 classification datasets · 21 classification datasets, tuned learning rates | mean normalized 0-1 loss: 0.1694 -> 0.1872 | Hurt |
| Replace tuned learning-rate baselines with parameter-free CODE | Optimization on regression and classification datasets · 17 regression and 21 classification datasets | normalized test loss: ~0.0740 on regression problems; ~0.0178 on classification problems | No clear effect |
| Use CODE rather than step-size-sensitive SGD, Adam, or AdaGrad | Synthetic absolute-loss regression · Synthetic dataset, noiseless and noisy regimes | iterations to reach 0.05 suboptimality gap | Improved |
| Use CODE rather than Coin | Parameter-free optimization on synthetic regression · Synthetic dataset, noiseless regime | iterations to reach 0.05 suboptimality gap | Improved |
Closest papers by meaning
- Training Deep Networks without Learning Rates Through Coin Betting (NeurIPS 2017 · similarity 0.88)
- Coin Betting and Parameter-Free Online Learning (NeurIPS 2016 · similarity 0.88)
- Tuning-Free Stochastic Optimization (ICML 2024 · similarity 0.87)
- How Free is Parameter-Free Stochastic Optimization? (ICML 2024 · similarity 0.85)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.84)
- Tighter Convergence Bounds for Shuffled SGD via Primal-Dual Perspective (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
