Research map / Optimization / Optimizers
Learning Step Size Controllers for Robust Neural Network Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace static step size with learned step size controller | RMSprop · MNIST full training error | training error | Helped |
| replace static step size with learned step size controller | SGD with momentum · MNIST and CIFAR-10 training error | training error | Helped |
| vary initial step size eta0 for controlled training (ablation) | controlled RMSprop · MNIST/CIFAR-10 | asymptotic training error | No clear effect |
| perturb learned controller parameters by 20% (ablation) | controlled RMSprop · MNIST | training error | No clear effect |
| vary discount factor gamma in [0.5, 0.99] (ablation) | controller features · MNIST | training error | No clear effect |
| account for 36% compute overhead of the controller | controlled RMSprop · MNIST validation error vs wall clock | validation error | Helped |
| reuse controller trained for RMSprop on SGD or other algorithms | learned controller · cross-algorithm transfer | generalization | Hurt |
Closest papers by meaning
- Special Properties of Gradient Descent with Large Learning Rates (ICML 2023 · similarity 0.85)
- Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size (NeurIPS 2025 · similarity 0.82)
- Two‑Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion (NeurIPS 2025 · similarity 0.82)
- Deep learning for continuous-time stochastic control with jumps (NeurIPS 2025 · similarity 0.82)
- FESSNC: Fast Exponentially Stable and Safe Neural Controller (ICML 2024 · similarity 0.82)
- Learning Robust Neural Processes with Risk-Averse Stochastic Optimization (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.