AI Paper Atlas

Research map / Federated learning and privacy / Communication-efficient FL

Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression: Fast Convergence and Partial Participation

Xiaoyun Li, Ping Li

ICML 2023 · Federated learning and privacy › Communication-efficient FL · approach: Federated optimization algorithms

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add error feedback to Fed-EF-SGD (Sign/TopK/Heavy-Sign compressors)Fed-SGD with biased compression · MNIST p=0.5 and p=0.1test accuracy (%)Improved
Add error feedback to Fed-EF-AMS on FMNIST at p=0.1 (ablation)AMSGrad global optimizer with Sign compression · FMNIST p=0.1test accuracy (%): 57.60 -> 64.09Improved
Use TopK compressor instead of sign-based compressors in Fed-EF-AMSFed-EF-AMS on FMNIST · FMNIST p=0.1 vs p=0.5test accuracy (%): TopK 50.77 vs Sign 64.09 at p=0.1; TopK 70.73 vs Sign 71.90 at p=0.5Mixed
Use Sign compressor with Fed-EF-AMS instead of full precision (p=0.5, MNIST)Fed-EF-AMS · MNIST p=0.5test accuracy (%): 92.74 vs 92.23No clear effect
Use unbiased stochastic quantization without EF instead of Fed-EF-SGD with heavy-SignFed-SGD communication compression · MNIST p=0.5test accuracy (%): 90.85 vs 90.16 (Hv-Sign)Hurt
Error restarting (reset error accumulator if stale for S=10 rounds) under partial participation (ablation)Fed-EF under PP with TopK-0.01 · MNIST and FMNIST p=0.1, logistic regression and MLPsquared gradient normImproved
Increase participation from m=20 to m=100 vs increase n under full participationFed-EF-SGD convergence · MNIST MLPsquared gradient normImproved
Partial participation (p=0.1) instead of p=0.5 in Fed-EF-SGDFed-EF-SGD · MNIST and FMNISTtest accuracy (%): MNIST full-precision 90.87 (p=0.5) vs 90.15 (p=0.1); FMNIST 71.20 vs 67.50Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy