AI Paper Atlas

Research map / Optimization / Nonconvex and smooth optimization

Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks

Naoki Sato, Koshiro Izumi, Hideaki Iiduka

JMLR 2024 · Optimization › Nonconvex and smooth optimization · approach: Stochastic variance reduction · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use constant learning rates instead of diminishing learning rates (ablation)SCG methods · CIFAR-100/CIFAR-10 ResNet-18 and IMDb LSTMtraining loss and accuracyImproved
use SCGAdam instead of other optimizers with constant learning rateResNet-18 · CIFAR-100training lossImproved
use SCGAdam instead of other optimizers with constant learning rateResNet-18 · CIFAR-10training lossImproved
use Momentum instead of SCGAdamResNet-18 · CIFAR-100 (200 epochs)test accuracyHurt
use SCGAdam instead of other optimizersResNet-18 · CIFAR-10epochs to reach 90% test accuracy: 90 %Improved
use SCGAdam or AMSGrad instead of other optimizersResNet-18 · CIFAR-100epochs to reach 70% test accuracy: 70 %Improved
use SCGAdam alongside RMSpropLSTM · IMDbtraining lossImproved
use SCGAMSGrad instead of Adam-type baselinesDCGAN · LSUN-BedroomFIDImproved
use SCGAMSGrad instead of other adaptive optimizersSNGAN · CIFAR10FIDImproved
use SCGAdam instead of other adaptive optimizersWGAN-GP · Tiny ImageNetFIDImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy