Research map / Federated learning and privacy / Communication-efficient FL

Clustered Federated Learning via Gradient-based Partitioning

ICML 2024 · Federated learning and privacy › Communication-efficient FL · approach: Local drift correction

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CFL-GP, a clustered federated learning method that accumulates client gradients over multiple models and clusters them with spectral clustering, with convergence guarantees for both clustering and models.

Problem

Clustered federated learning under privacy-preserving protocols (no raw data sharing) where existing methods often fail to find correct client groupings, leading to sub-optimal performance.

Approach

Periodically compute client gradients over a round-robin set of models, accumulate them with cumulative averaging into per-client feature vectors, and cluster clients via spectral clustering (SVD dimensionality reduction plus k-means); the accumulation makes clustering accuracy improve monotonically over time and yields provable convergence without good initialization.

Key results

• CFL-GP achieves ARI 1.00 on rotated MNIST, CIFAR-10 (ResNet-18), and EMNIST across client counts, while IFCA and MADMO degrade at larger client counts (e.g., MADMO ARI 0.00 at C=80 on CIFAR-10).
• On the COST2100 deep autoencoder task, CFL-GP reaches -19.38 dB NMSE versus -15.83 dB for IFCA and -17.63 dB for MADMO (C=16).
• The paper reports clustering up to 100 times faster than existing state-of-the-art CFL algorithms.

Measured effects reported in the paper

Surprising findings

• On CIFAR-10 with ResNet-18, IFCA matches CFL-GP in accuracy (73.97 vs 73.93 at C=20), differing mainly in clustering consistency for larger client counts.
• Compressing gradients by more than 100x still allowed effective spectral clustering.

Limitations

• Theoretical analysis assumes Gaussian stochastic gradient noise and the number of distributions equals the number of models K.
• Number of clusters K must be fixed in advance; the paper proposes a separate method for unknown K in an appendix.

Experimental setup

Datasets: MNIST CIFAR-10 EMNIST COST2100 synthetic linear regression data

Benchmarks: MNIST rotated CIFAR-10 EMNIST COST2100 synthetic multiple linear regression

Metrics: Adjusted Rand Index classification accuracy MSE NMSE communication cost computation cost

Baselines: IFCA MADMO FedAvg PACFL FedEM

Builds on: IFCA FedAvg MADMO PACFL

Closest papers by meaning

• Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies (ICLR 2025 · similarity 0.92)
• Interaction-Aware Gaussian Weighting for Clustered Federated Learning (ICML 2025 · similarity 0.90)
• EBS-CFL: Efficient and Byzantine-robust Secure Clustered Federated Learning (AAAI 2025 · similarity 0.90)
• FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.89)
• Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.88)
• Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable (NeurIPS 2025 · similarity 0.87)