AI Paper Atlas

Research map / Optimization / Optimizers

Probabilistic Line Searches for Stochastic Optimization

Maren Mahsereci, Philipp Hennig

JMLR 2017 · Optimization › Optimizers · approach: Differentiating through embedded optimization · 81 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
control SGD step size with the probabilistic line search instead of a fixed learning rateSGD on 2-layer NN · MNIST and CIFAR-10 test error after 10 epochstest error vs initial learning rate: initial learning rate across five orders of magnitude gives close to optimal errorImproved
add line search to SGD compared with best tuned fixed-rate SGD without decaySGD · MNIST and CIFAR-10test error: best SGD instance just barely outperformed the line searchesNo clear effect
pick learning rate from short exploratory experiment (10^3 steps) rather than line search with alpha0=1SGD · MNIST and CIFAR-10test error: alpha=0.75 (MNIST) and alpha=0.08 (CIFAR-10) best after 10^3 steps but not best after 10 epochsHurt
add line search to SGD with large step sizesSGD · MNIST and CIFAR-10 training losstraining stabilityImproved
add line search to SGD, effect on overfittingSGD · CIFAR-10 and MNIST test vs train erroroverfittingNo clear effect
Wolfe-condition based line search, line search length statisticsProbabilistic line search · MNISTfunction evaluations per line search: ~1.4 function evaluations on average, 80%-90% terminate after first evaluationImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy