Research map / Federated learning and privacy / Communication-efficient FL
FADAS: Towards Federated Adaptive Asynchronous Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FADAS, an asynchronous adaptive federated optimizer (FedBuff-style buffered updates with AMSGrad-like server step) plus a delay-adaptive learning rate, with convergence guarantees and experiments on CIFAR-10/100 (ResNet-18) and GLUE (BERT + LoRA).
Problem
Synchronous adaptive federated optimization waits for stragglers, and the effect of asynchronous gradient delays on nonlinear adaptive server updates is not well understood.
Approach
Combine asynchronous buffered client updates (FedBuff-style concurrency and buffer size) with an Adam/AMSGrad-style global adaptive update on the pseudo-gradient, and further scale the global learning rate by the current maximum delay when it exceeds a threshold.
Key results
- Under large worst-case delay on CIFAR-10 (Dir 0.1), delay-adaptive FADAS reaches 73.96% vs FedBuff 38.68% and FedAsync 50.92%.
- Under mild delay on CIFAR-100 (Dir 0.1), FADAS reaches 57.37% vs FedBuff 53.70% and FedAsync 45.26%.
- Simulated wall-clock time on CIFAR-10 to 75% accuracy: FADAS 228.0 vs FedAMS 648.7 and FedAvg 2257.7 (units of 10 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use asynchronous adaptive aggregation with FADAS instead of the strongest asynchronous SGD baseline | FADAS · CIFAR-10 and CIFAR-100, large worst-case and mild delays; RTE, MRPC, and SST-2 mild delay | accuracy | Helped |
| add delay-adaptive learning rate | FADAS · CIFAR-10 and CIFAR-100, large worst-case delay | accuracy | Helped |
| add delay-adaptive learning rate | FADAS · CIFAR-10 and RTE, mild delay | accuracy | Helped |
| add delay-adaptive learning rate | FADAS · CIFAR-100, MRPC, and SST-2, mild delay | accuracy | Hurt |
| set delay threshold to $\tau_c=8$ rather than $\tau_c=1$, $4$, or $10$ (ablation) | delay-adaptive FADAS · CIFAR-10, large worst-case delay | accuracy | Helped |
| add delay-adaptive learning rate at concurrency $M_c=15$ or $M_c=25$ (ablation) | FADAS · CIFAR-10, large worst-case delay | accuracy | Helped |
| add delay-adaptive learning rate at concurrency $M_c=35$ (ablation) | FADAS · CIFAR-10, large worst-case delay | accuracy | Hurt |
| increase buffer size $M$ (ablation) | FADAS and delay-adaptive FADAS · CIFAR-10, large worst-case delay | accuracy | Helped |
| use smaller buffer size $M=3$ instead of larger buffer sizes (ablation) | FADAS and delay-adaptive FADAS · CIFAR-10, large worst-case delay | training time to target accuracy | Helped |
Surprising findings
- FedAsync shows severely fluctuating test accuracy under large worst-case delays.
- Delay-adaptive FADAS gives no clear gain under mild delays and is slightly worse on some settings (e.g., CIFAR-100 mild delay, RTE time-to-accuracy).
Limitations
- Convergence theory relies on uniform arrival assumption and bounded worst-case delay, used only for analysis.
- Wall-clock times are simulated with uniform distributions rather than measured on real systems.
- Delay-adaptive threshold tau_c appears to depend on the actual delay distribution and must be tuned.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 RTE MRPC SST-2
Benchmarks: CIFAR-10 CIFAR-100 RTE MRPC SST-2
Metrics: test accuracy wall-clock time to target accuracy
Baselines: FedAsync FedBuff FedAvg FedAMS
Builds on: FedBuff FedAsync FedAMS FedAdam AMSGrad
Closest papers by meaning
- Efficient Adaptive Federated Optimization (NeurIPS 2025 · similarity 0.89)
- FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · similarity 0.88)
- FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update (AAAI 2024 · similarity 0.88)
- Momentum-Driven Adaptivity: Towards Tuning-Free Asynchronous Federated Learning (ICML 2025 · similarity 0.88)
- Efficient Asynchronous Federated Learning with Prospective Momentum Aggregation and Fine-Grained Correction (AAAI 2024 · similarity 0.88)
- FedECADO: A Dynamical System Model of Federated Learning (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.