AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect

Yuqing Wang, Minshuo Chen, Tuo Zhao, Molei Tao

ICLR 2022 · Optimization › Implicit bias and generalization · approach: Implicit regularization of SGD · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use a sufficiently large constant learning rate instead of an infinitesimal or diminishing learning rateGradient Descent on homogeneous matrix factorization · General matrix factorizationfactor imbalanceImproved
Use large learning rate on an initially unbalanced scalar factorizationGradient Descent · Scalar factorization, μ=1, ||x₀||=18, ||y₀||=0.09factor difference: 17.9986 to 0.0154Improved
Use larger learning rates within the convergence rangeGradient Descent on scalar factorization · Scalar factorizationfactor norm gapImproved
Use small learning rates rather than large learning ratesGradient Descent on scalar factorization · Scalar factorizationfactor norm gapNo clear effect
Use learning rates above 2/LGradient Descent on scalar factorization · Scalar factorizationloss monotonicityHurt
Use learning rates above 2/L within the analyzed rangeGradient Descent on scalar factorization · Scalar factorizationconvergence to a global minimumImproved
Use learning rates that are too largeGradient Descent on matrix factorization · General matrix factorizationconvergenceHurt
Use larger learning rates near the stability limitGradient Descent · Matrix sensingfactor norm balanceImproved
Use larger learning ratesGradient Descent · Matrix completionfactor norm gapImproved
Use step size h=1.9Gradient Descent on (1-xy)²/2 · Scalar factorizationconvergence to a fixed global minimumHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy