AI Paper Atlas

Research map / Optimization / Optimizers

MLR-SNet: Transferable LR Schedules for Heterogeneous Tasks

Jun Shu, Yanwen Zhu, Qian Zhao, Deyu Meng, Zongben Xu

TPAMI 2022 · Optimization › Optimizers · approach: Memory-efficient LLM optimizers · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace the MultiStep schedule with a meta-learned MLR-SNet scheduleSGD · CIFAR-100 (SGD base)test accuracy: 77.04 -> 80.44Improved
replace the MultiStep schedule with MLR-SNetSGD · CIFAR-10 (SGD base)test accuracy: 93.82 -> 94.80Improved
replace SGDM+MultiStep with SGD driven by MLR-SNetSGDM+MultiStep · CIFAR-10test accuracy: 95.08 -> 94.80No clear effect
replace validation-based LR dropping with MLR-SNetSGD+Val Strategy · Penn Treebank 3-layer LSTMtest perplexity: 76.05 -> 72.28Improved
replace validation-based LR dropping with MLR-SNetSGD+Val Strategy · Penn Treebank 2-layer LSTMtest perplexity: 74.33 -> 70.53Improved
replace hypergradient-descent LR adaptation with MLR-SNetSGD+HD · CIFAR-100test accuracy: 72.22 -> 80.44Improved
use MLR-SNet instead of SGDR when training on corrupted CIFAR-10-CSGDR · CIFAR-10-C (last epoch)test accuracy: 78.21 -> 85.87Improved
use an LSTM meta-learner instead of an MLP meta-learner (ablation)MLR-SNet · CIFAR-10 / Penn Treebanktest performanceImproved
combine several saved MLR-SNets (k>=3) rather than a single net at meta-test (ablation)MLR-SNet · CIFAR-100 meta-testtest accuracyImproved
change depth or hidden-node size of MLR-SNet (ablation)MLR-SNet · CIFAR-10 ResNet-18test accuracyNo clear effect
vary the global LR of the Adam meta-optimizer (ablation)MLR-SNet · CIFAR-10 ResNet-18test accuracyNo clear effect
meta-train on more similar tasks (MNIST, SVHN, CIFAR-10) (ablation)MLR-SNet · CIFAR-100 meta-testtest accuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy