Research map / Optimization / Optimizers
Stab-SGD: Noise-Adaptivity in Smooth Optimization with Stability Ratios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the stability ratio of gradient estimates as a local relative noise measure and proposes Stab-SGD, a schedule-free SGD variant that shrinks the learning rate by this ratio to adapt to noise levels.
Problem
Smooth stochastic optimization with first-order methods where the noise level varies and standard SGD requires noise-dependent tuning of learning rate schedules.
Approach
Defines a stability ratio of gradient estimates (from zero for pure noise to one for negligible noise) and uses it to scale the learning rate of a schedule-free SGD, yielding noise adaptivity without a tuned schedule.
Key results
- Abstract states Stab-SGD obtained by shrinking the learning rate by the stability ratio achieves real adaptivity to noise levels (abstract truncated; no numeric results available).
Limitations
- Full text unavailable; only abstract content could be used.
Experimental setup
Builds on: stochastic gradient descent schedule-free optimization
Closest papers by meaning
- Tuning-Free Stochastic Optimization (ICML 2024 · similarity 0.91)
- Error Analysis Affected by Heavy-Tailed Gradients for Non-Convex Pairwise Stochastic Gradient Descent (AAAI 2025 · similarity 0.91)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.89)
- Generalization Analysis of Biased Stochastic Gradient Methods for Minimax Problems (NeurIPS 2026 · similarity 0.89)
- Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters (NeurIPS 2026 · similarity 0.89)
- Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.