AI Paper Atlas

Research map / Optimization / Distributed and compressed

GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models

Wenbo Su, Yuanxing Zhang, Yufeng Cai, Kaixu Ren, Pengjie Wang, Huimin Yi, Yue Song, Jing Chen, Hongbo Deng, Jian Xu, Lin Qu, Bo Zheng

NeurIPS 2022 · Optimization › Distributed and compressed · approach: Local and asynchronous SGD · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use GBA instead of the best baseline (Hop-BW) when switching from synchronous trainingGBA · Criteo, Alimama, Privateaverage AUC: at least 0.2% on averageImproved
Use GBA instead of other modes when switching to synchronous trainingGBA · three tasksAUC: at least 0.1%Improved
Use GBA instead of synchronous training under strained hardwareGBA · three taskstraining speed: at least 2.4xImproved
Use GBA instead of synchronous AR trainingGBA · Criteoglobal QPS: 1,436K($\pm$224K) -> 3,240K($\pm$97K)Improved
Use GBA instead of synchronous AR trainingGBA · Privateglobal QPS: 43K($\pm$21K) -> 87K($\pm$19K)Improved
Use GBA instead of canonical asynchronous trainingGBA · Criteoglobal QPS: 3,253K($\pm$84K) -> 3,240K($\pm$97K)No clear effect
Use GBA instead of Hop-BSGBA · Privateglobal QPS: 29K($\pm$11K) -> 87K($\pm$19K)Improved
Use GBA instead of Hop-BW (backup workers)GBA · Privateexcluded batches: 300K -> 786Improved
Use GBA instead of BSP for gradient aggregationGBA · Privateavg gradient staleness: 2.61 (12) -> 0.21 (11)Improved
Run GBA in different cluster periods of the dayGBA · PrivateAUC: 0.7864 -> 0.7866No clear effect
Vary number of workers from 100 to 800 while keeping global batch size (ablation)GBA · PrivateAUC and QPS: AUC difference less than 10^-4Mixed
Use a global batch size different from synchronous training (400 workers, vary local batch size) (ablation)GBA · PrivateAUC after switchingHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy