AI Paper Atlas

Research map / Optimization / Distributed and compressed

DecentLaM: Decentralized Momentum SGD for Large-Batch Deep Training

Kun Yuan, Yiming Chen, Xinmeng Huang, Yingya Zhang, Pan Pan, Yinghui Xu, Wotao Yin

ICCV 2021 · Optimization › Distributed and compressed · approach: Local and asynchronous SGD · 36 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use DecentLaM instead of DmSGD at batch 32kdecentralized momentum SGD · ImageNet-1K val (batch 32k)top-1 accuracy (%): 74.97 -> 76.22Improved
use DecentLaM instead of DmSGD at batch 2kdecentralized momentum SGD · ImageNet-1K val (batch 2k)top-1 accuracy (%): 76.27 -> 76.43No clear effect
use DecentLaM instead of parallel PmSGD at batch 32kPmSGD · ImageNet-1K val (batch 32k)top-1 accuracy (%): 75.27 -> 76.22Improved
increase batch size from 2K to 32K in DmSGD without LARSDmSGD · ImageNet-1K val (Table 1 setup)top-1 accuracy (%): 76.5% -> 74.9%Hurt
increase batch size from 2K to 8K in DmSGD on CIFAR-10DmSGD · CIFAR-10 (ResNet-20)top-1 accuracy (%): 91.5% -> 88.3%Hurt
increase batch size from 16k to 32k in DecentLaMDecentLaM · ImageNet-1K val (ResNet-50)top-1 accuracy (%): 76.73 -> 76.22Hurt
use ring instead of symmetric exponential topology at batch 32k (ablation)DecentLaM · ImageNet-1K val (ResNet-50, batch 32k)top-1 accuracy (%): 76.22 -> 76.34Improved
use mesh instead of symmetric exponential topology at batch 32k (ablation)DecentLaM · ImageNet-1K val (ResNet-50, batch 32k)top-1 accuracy (%): 76.22 -> 76.47No clear effect
use bipartite random match instead of symmetric exponential topology at batch 16k (ablation)DecentLaM · ImageNet-1K val (ResNet-50, batch 16k)top-1 accuracy (%): 76.73 -> 76.53No clear effect
use DecentLaM instead of DmSGD on COCO with Faster R-CNNDmSGD · COCOmAP: 36.4 -> 37.1Improved
use DecentLaM instead of DmSGD on PASCAL VOC with RetinaNetDmSGD · PASCAL VOCmAP: 79.1 -> 79.3No clear effect
use DecentLaM instead of D2-DmSGD at batch 2kdecentralized momentum SGD · ImageNet-1K val (batch 2k)top-1 accuracy (%): 75.44 -> 76.43Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy