AI Paper Atlas

Research map / Optimization / Optimizers

Towards Better Generalization of Adaptive Gradient Methods

Yingxue Zhou, Belhal Karimi, Jinxing Yu, Zhiqiang Xu, Ping Li

NeurIPS 2020 · Optimization › Optimizers · approach: Adam convergence and adaptive learning rates · 7 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use SAGD instead of RMSprop/Adam/AdaBound/SGDLSTM optimizer · Penn Treebank 2-layer LSTMtest perplexity: 65.96 -> 60.66Improved
Use SAGD instead of best baseline AdaBoundLSTM optimizer · Penn Treebank 3-layer LSTMtest perplexity: 62.33 -> 59.43Improved
Use SAGD instead of Adam/RMSpropLSTM optimizer · Penn Treebank 3-layer LSTMoverfitting after 150 epochsImproved
Use SAGD instead of RMSprop/Adam2-layer LSTM · Penn Treebanktrain perplexityHurt
Use SAGD instead of baselines2-layer ReLU network · MNISTtest accuracyImproved
Use SAGD instead of baselines2-layer Sigmoid network · MNISTtest accuracyImproved
Use SAGD instead of baselinesResNet-18 · CIFAR-10test accuracyImproved
Use SAGD instead of adaptive baselinesVGG-19 · CIFAR-10test lossMixed
Use SAGD instead of baselinesbiLSTM · SNLItrain vs test perplexityMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy