Research map / Learning theory and algorithms / Kernels and regression theory
Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
Jihwan Kim, Dogyoon Song, Chulhee Yun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SGD with signSGD | linear PLRF model · PLRF (alpha=0.4, beta=0.8) | compute-optimal slope | Improved |
| replace SGD with signSGD | linear PLRF model · PLRF (alpha=0.7, beta=1.1) | compute-optimal slope | Improved |
| replace SGD with AdamW | Transformer · OpenWebText | compute-optimal loss exponent: -0.005 -> -0.021 | Improved |
| add warmup-stable-decay schedule instead of constant learning rate | signSGD · PLRF (alpha=1.0, beta=0) | compute-optimal slope | Improved |
| replace SGD with signSGD in regimes with small noise bottleneck | linear PLRF model · PLRF phases Aa, Ab, Ac, Ba, I, II | compute-optimal slope | No clear effect |
| use maximal instead of optimal learning rate (ablation) | signSGD · PLRF | compute-optimal slope | Hurt |
Closest papers by meaning
- Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects (ICML 2025 · similarity 0.89)
- When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds (NeurIPS 2026 · similarity 0.89)
- Investigating the Role of Weight Decay in Enhancing Nonconvex SGD (CVPR 2025 · similarity 0.88)
- Scaling Laws in Linear Regression: Compute, Parameters, and Data (NeurIPS 2024 · similarity 0.87)
- Improved Scaling Laws in Linear Regression via Data Reuse (NeurIPS 2025 · similarity 0.86)
- The Optimization Landscape of SGD Across the Feature Learning Strength (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
