AI Paper Atlas

Research map / Optimization / Optimizers

Asynchronous Stochastic Gradient Descent for Extreme-Scale Recommender Systems

Lewis Liu, Kun Zhao

AAAI 2021 · Optimization › Optimizers · approach: Memory-efficient LLM optimizers · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Apply normalized staleness decay to asynchronous gradientsASGD for the Wide & Deep variant · E-commerce CTR, 3K workersAUCImproved
Use linear staleness normalization instead of the proposed normalized decayASGD for the Wide & Deep variant · E-commerce CTR, 3K workersAUCHurt
Add data normalization to staleness normalizationASGD for the Wide & Deep variant · E-commerce CTR, 1K workersAUC: improves the result by 1%Improved
Add data normalization to staleness normalizationASGD for the Wide & Deep variant · E-commerce CTR, 3K workersAUC: improves the result by 2%Improved
Remove data normalization (ablation)Staleness-normalized ASGD · E-commerce CTR, 1K and 3K workersAUC: 1%; 2%Hurt
Use Adagrad-SWAP instead of original AdagradWide & Deep variant · E-commerce CTR, 10 days of incremental trainingAUCImproved
Use original Adagrad without SWAP (ablation)Wide & Deep variant · E-commerce CTR, 10 days of incremental trainingAUCHurt
Decay Adagrad accumulators at every step instead of periodically (ablation)Wide & Deep variant · E-commerce CTR, 10 days of incremental trainingAUCNo clear effect
Scale asynchronous training from 1K to 3K workersASGD without the proposed staleness normalization · E-commerce CTRAUCHurt
Scale asynchronous training from 1K to 3K workers while using staleness normalizationASGD with staleness normalization · E-commerce CTRAUCNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy