AI Paper Atlas

Research map / Optimization / Optimizers

Small Steps and Giant Leaps: Minimal Newton Solvers for Deep Learning

Joao F. Henriques, Sebastien Ehrhardt, Samuel Albanie, Andrea Vedaldi

ICCV 2019 · Optimization › Optimizers · approach: Natural gradient and Fisher approximations · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use CurveBall instead of SGDVGG-f · ImageNet-100best validation error (%): 39.8 -> 33.8Improved
use CurveBall instead of AdamVGG-f · ImageNet-100best validation error (%): 35.9 -> 33.8Improved
use CurveBall instead of SGDResNet-18 · CIFAR-10best validation error (%): 12.8 -> 16.1Hurt
add dropout 0.3 to CurveBall lambdaResNet-18 · CIFAR-10validation error (%): 15.3 -> 13.5Improved
add automatic lambda rescaling (ablation)CurveBall · CIFAR-10 basic CNNtraining error (%): 15.3 -> 14.1Improved
add automatic lambda rescaling (ablation)CurveBall · CIFAR-10 basic CNN + BNtraining error (%): 9.4 -> 7.6Improved
add automatic lambda rescaling (ablation)CurveBall · CIFAR-10 ResNet-18training error (%): 1.3 -> 0.7Improved
add automatic lambda rescaling (ablation)CurveBall · ImageNet-100 VGG-ftraining error (%): 12.7 -> 10.3Improved
add automatic lambda rescaling (ablation)CurveBall · CIFAR-10 basic CNN + BNvalidation error (%): 15.8 -> 16.3Hurt
add batch normalisationCurveBall · CIFAR-10 basic CNNbest validation error (%): 19.3 -> 15.8Improved
use CurveBall instead of SGDBasic CNN · CIFAR-10training error (%): 18.9 -> 15.3Improved
use CurveBall instead of Adam on the noisy Rosenbrock U[0,3]CurveBall · Stochastic Rosenbrock U[0,3]iterations to solution: 2750 -> 13Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy