Research map / Federated learning and privacy / Communication-efficient FL
Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HiCS-FL, a client selection method that estimates each client's data heterogeneity from the output-layer bias update, clusters clients with a heterogeneity-aware distance, and samples clusters by heterogeneity, with convergence analysis and experiments on FMNIST, CIFAR10, Mini-ImageNet and THUC news.
Problem
Client selection in federated learning under non-IID data and partial participation: existing methods are computationally expensive or fail to distinguish clients with balanced versus imbalanced label distributions.
Approach
Estimate client label-distribution entropy from the softmax of the output-layer bias update (theoretically linked to label proportions under CE loss), then cluster clients with a distance combining gradient cosine distance and entropy difference, and sample clusters with probability favoring higher average entropy (annealed over rounds).
Key results
- On FMNIST, HiCS-FL reaches 0.75 test accuracy in 60 rounds vs 149 for random sampling (2.5x speedup).
- On CIFAR10, HiCS-FL reaches 0.6 accuracy in 123 rounds vs 898 for random sampling (7.3x speedup).
- On THUC news, HiCS-FL achieves the highest test accuracy in all three partition settings (83.2%, 83.9%, 79.7%) versus CS and FedCor.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HiCS-FL instead of pow-d client sampling | FedAvg with CNN · FMNIST (setting 2), test accuracy = 0.75 | communication rounds: 79 -> 60 | Helped |
| Use HiCS-FL instead of FedCor client sampling | CNN federated learning · CIFAR10 (setting 2), test accuracy = 0.6 | communication rounds: 711 -> 123 | Helped |
| Use HiCS-FL instead of CS client sampling | ResNet18 federated learning · Mini-ImageNet (setting 2), test accuracy = 0.5 | communication rounds: 186 -> 86 | Helped |
| Use HiCS-FL instead of CS client sampling | TextRNN federated learning · THUC news (setting 2), test accuracy = 0.8 | communication rounds: 74 -> 27 | Helped |
| Use HiCS-FL instead of FedCor client sampling | TextRNN · THUC news setting (1) | test accuracy: 81.2 -> 83.2 | Helped |
| Use HiCS-FL instead of CS client sampling | TextRNN · THUC news setting (2) | test accuracy: 82.8 -> 83.9 | Helped |
| Use HiCS-FL instead of CS client sampling | TextRNN · THUC news setting (3) | test accuracy: 79.4 -> 79.7 | Helped |
Surprising findings
- DivFL is slower than random sampling on most tasks in the reported speedup table (e.g., 0.3x on FMNIST).
- Clustered Sampling outperforms Power-of-Choice and DivFL on THUC news test accuracy in settings 2 and 3.
Limitations
- Theory relies on SGD and cross-entropy assumptions; extension to Adam/FedProx is shown empirically only.
- Speedup depends on experimental partition settings with hand-picked Dirichlet alpha values and a mix of balanced and imbalanced clients.
- Requires tuning of hyper-parameters such as temperature T, lambda, and gamma0.
Experimental setup
Datasets: FMNIST CIFAR10 Mini-ImageNet THUC news
Benchmarks: FMNIST CIFAR10 Mini-ImageNet THUC news
Metrics: test accuracy average training loss rounds to target accuracy speedup vs random sampling
Baselines: Random sampling Power-of-Choice (pow-d) Clustered Sampling DivFL FedCor FedProx
Builds on: FedAvg FedProx Clustered Sampling Power-of-Choice SCAFFOLD
Closest papers by meaning
- Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.92)
- Ranking-based Client Imitation Selection for Efficient Federated Learning (ICML 2024 · similarity 0.89)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.89)
- Reinforcement Active Client Selection for Federated Heterogeneous Graph Learning (AAAI 2025 · similarity 0.89)
- Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.88)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.