AI Paper Atlas

Research map / Federated learning and privacy / Communication-efficient FL

Workie-Talkie: Accelerating Federated Learning by Overlapping Computing and Communications via Contrastive Regularization

Rui Chen, Qiyu Wan, Pavana Prakash, Lan Zhang, Xu Yuan, Yanmin Gong, Xin Fu, Miao Pan

ICCV 2023 · Federated learning and privacy › Communication-efficient FL · approach: Efficient local training in FL · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use class-wise temperature instead of fixed temperature in contrastive regularization (ablation)FedCR · CIFAR-100 (beta=0.5)test accuracy: 61.3% -> 62.3%Improved
use class-wise temperature instead of average temperature (ablation)FedCR · CIFAR-100 (beta=0.5)test accuracy: 61.9% -> 62.3%Improved
class-wise temperature gain grows as data heterogeneity increases (beta smaller) (ablation)FedCR · CIFAR-10/CIFAR-100test accuracy: CIFAR-10 beta=0.5: 82.2% -> 83.1%; beta=5: 87.2% -> 88.2%Improved
contrastive regularization with fixed projector instead of gradient correction (DGA)overlapped FL · CIFAR-100test accuracy: 58.8% -> 62.3%Improved
replace trainable projector (Overlap-MOON) with fixed non-trainable projectorFedCR · CIFAR-100test accuracy / speedup: 63.6% -> 62.3% accuracy; speedup 2.18 -> 2.58Mixed
overlap computing and communication with FedCR vs DGA/LGCFL training · CIFAR-100 target 58.8% accuracytraining time speedup: 1.52x vs DGA, 2.01x vs LGCImproved
FedCR vs DGA/LGC rounds on Tiny-ImageNetFL training · Tiny-ImageNet target 20%communication rounds: 1.65x and 2.67x fewer rounds vs DGA and LGCImproved
increase staleness level (DGA gradient correction) (ablation)DGA · CIFAR-10 ResNet20 CKACKA similarity local vs global: 0.72 -> 0.51Hurt
train heterogeneous subnetworks with FedCR instead of Overlap-HeteroFLFedCR · CIFAR-100 p-subnetworkstest accuracy: +23%, +15%, +37% on different p-subnetworksImproved
FedCR vs Overlap-MOON with more devices and time budgetFedCR · CIFAR-100 N=64, 25x10^2 stest accuracy: 58.01% -> 64.68%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy