Research map / Federated learning and privacy / Communication-efficient FL

Hierarchical Federated Learning with Multi-Timescale Gradient Correction

NeurIPS 2024 · Federated learning and privacy › Communication-efficient FL · approach: Local drift correction · 11 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MTGC, a hierarchical federated learning method that uses two coupled control variables (client-group and group-global gradient corrections) updated at different timescales, with a convergence bound independent of data heterogeneity.

Problem

Hierarchical federated learning suffers from multi-timescale model drift under multi-level non-i.i.d. data, and existing HFL convergence bounds degrade as heterogeneity grows, while star-topology drift-correction methods do not extend easily to hierarchies.

Approach

Introduce two control variables: a client-group correction z updated after every group aggregation and a group-global correction y updated after every global aggregation, which correct client gradients toward group gradients and group gradients toward the global gradient; the convergence analysis handles their coupling with a Lyapunov function, and recovers SCAFFOLD when N=E=1.

Key results

• MTGC obtains the best testing accuracy over FedProx, SCAFFOLD, FedDyn and HFedAvg on four datasets under group non-i.i.d. and client non-i.i.d. settings (3 random trials).
• Theory: convergence rate of order sqrt(F0 L sigma^2 / (N~ T E H)) plus higher-order terms, with no bounded gradient dissimilarity assumption, giving linear speedup in E, H and number of clients.
• Communication overhead is (E+1)/E times that of HFedAvg per aggregation, since correction initialization is only needed once per global round.

Measured effects reported in the paper

Surprising findings

• FedDyn performs worst in the hierarchical setting, with large variance, suggesting star-topology methods may not transfer to HFL.

Limitations

• Experiments are limited to small-to-medium image and text models (MLP, CNN, ResNet-18, LSTM) on standard benchmarks.
• Assumes L-smooth losses and bounded stochastic gradient variance; extension to the three-level case is shown only empirically and without full theory in the main analysis.

Experimental setup

Datasets: EMNIST-Letters Fashion-MNIST CIFAR-10 CIFAR-100 CINIC-10 Shakespeare

Benchmarks: EMNIST-Letters Fashion-MNIST CIFAR-10 CIFAR-100 CINIC-10 Shakespeare

Metrics: testing accuracy convergence bound on average squared gradient norm

Baselines: HFedAvg FedProx SCAFFOLD FedDyn ProxSkip-HUB

Builds on: FedAvg SCAFFOLD hierarchical FedAvg gradient tracking

Closest papers by meaning

• FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.88)
• Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.88)
• FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.87)
• Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.87)
• Communication-Efficient Federated Learning with Accelerated Client Gradient (CVPR 2024 · similarity 0.87)
• FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler (ICML 2024 · similarity 0.87)