Research map / Optimization / Optimizers
MLR-SNet: Transferable LR Schedules for Heterogeneous Tasks
Jun Shu, Yanwen Zhu, Qian Zhao, Deyu Meng, Zongben Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the MultiStep schedule with a meta-learned MLR-SNet schedule | SGD · CIFAR-100 (SGD base) | test accuracy: 77.04 -> 80.44 | Improved |
| replace the MultiStep schedule with MLR-SNet | SGD · CIFAR-10 (SGD base) | test accuracy: 93.82 -> 94.80 | Improved |
| replace SGDM+MultiStep with SGD driven by MLR-SNet | SGDM+MultiStep · CIFAR-10 | test accuracy: 95.08 -> 94.80 | No clear effect |
| replace validation-based LR dropping with MLR-SNet | SGD+Val Strategy · Penn Treebank 3-layer LSTM | test perplexity: 76.05 -> 72.28 | Improved |
| replace validation-based LR dropping with MLR-SNet | SGD+Val Strategy · Penn Treebank 2-layer LSTM | test perplexity: 74.33 -> 70.53 | Improved |
| replace hypergradient-descent LR adaptation with MLR-SNet | SGD+HD · CIFAR-100 | test accuracy: 72.22 -> 80.44 | Improved |
| use MLR-SNet instead of SGDR when training on corrupted CIFAR-10-C | SGDR · CIFAR-10-C (last epoch) | test accuracy: 78.21 -> 85.87 | Improved |
| use an LSTM meta-learner instead of an MLP meta-learner (ablation) | MLR-SNet · CIFAR-10 / Penn Treebank | test performance | Improved |
| combine several saved MLR-SNets (k>=3) rather than a single net at meta-test (ablation) | MLR-SNet · CIFAR-100 meta-test | test accuracy | Improved |
| change depth or hidden-node size of MLR-SNet (ablation) | MLR-SNet · CIFAR-10 ResNet-18 | test accuracy | No clear effect |
| vary the global LR of the Adam meta-optimizer (ablation) | MLR-SNet · CIFAR-10 ResNet-18 | test accuracy | No clear effect |
| meta-train on more similar tasks (MNIST, SVHN, CIFAR-10) (ablation) | MLR-SNet · CIFAR-100 meta-test | test accuracy | No clear effect |
Closest papers by meaning
- AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly (ICLR 2021 · similarity 0.89)
- One LR Doesn’t Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs (ICML 2026 · similarity 0.86)
- Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks (ICLR 2018 · similarity 0.86)
- Subnet-Aware Dynamic Supernet Training for Neural Architecture Search (CVPR 2025 · similarity 0.85)
- A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks (NeurIPS 2024 · similarity 0.84)
- The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
