AI Paper Atlas

Research map / Optimization / Distributed and compressed

On the Trade-Off Between Flatness and Optimization in Distributed Learning

Ying Cao, Zhaoxian Wu, Kun Yuan, Ali H. Sayed

TPAMI 2025 · Optimization › Distributed and compressed · approach: Decentralized optimization algorithms

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use diffusion instead of centralized trainingcentralized SGD · CIFAR10 (from scratch, 512 x 16)test accuracy: 89.14 ± 0.41 -> 91.49 ± 0.08Improved
use diffusion instead of consensusconsensus · CIFAR100 (from scratch, 512 x 16)test accuracy: 67.38 ± 0.45 -> 69.74 ± 0.83Improved
use consensus instead of centralized trainingcentralized SGD · CIFAR100 (from scratch, 512 x 16)test accuracy: 68.65 ± 0.37 -> 67.38 ± 0.45Hurt
use consensus on ring instead of random graphconsensus · CIFAR100 (from scratch, 512 x 16)test accuracy: 67.38 ± 0.45 -> 64.97 ± 0.39Hurt
use consensus instead of diffusion when initialized from a pretrained centralized modeldiffusion · CIFAR100 (pretrained init, 512 x 16)test accuracy: 69.85 ± 0.18 -> 70.69 ± 0.33Improved
use consensus instead of centralized training with pretrained initializationcentralized SGD · CIFAR100 (pretrained init, 128 x 16)test accuracy: 70.20 ± 0.40 -> 72.08 ± 0.31Improved
use consensus instead of centralized training from scratchcentralized SGD · CIFAR10 (from scratch, 512 x 16)generalization gap: 0.1053 -> 0.0686Improved
use consensus instead of centralized training from scratchcentralized SGD · CIFAR100 (from scratch, 512 x 16)generalization gap: 0.3126 -> 0.2027Improved
use diffusion instead of centralized training from scratchcentralized SGD · CIFAR100 (from scratch, 512 x 16)generalization gap: 0.3126 -> 0.2977No clear effect
use pretrained initialization instead of random init for consensusconsensus · CIFAR100 (512 x 16, random graph)test accuracy: 67.38 ± 0.45 -> 70.69 ± 0.33Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy