Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Learning with Sample-level Client Drift Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedBSS, a two-stage federated learning method that warms up with standard aggregation and then progressively adds high-bias (high-loss, uncertainty-split) client samples epoch by epoch to mitigate sample-level client drift under non-IID data.
Problem
Federated learning under non-IID data suffers from client drift, but existing methods calibrate local updates or aggregation without understanding how individual heterogeneous samples contribute to drift.
Approach
Treat client drift as the cumulative effect of per-sample bias (measured by global-model loss); split each client's samples into unbiased and biased sets using an uncertainty-based threshold, and progressively include biased samples across local epochs with a cosine schedule, after a warm-up stage.
Key results
- On CIFAR-10 with Dirichlet 0.5, FedBSS with ResNet-50 reaches 45.27% accuracy versus 40.76% for the best baseline (FedLC).
- FedBSS achieves the best accuracy in most label-skew settings, e.g. 76.24% on Fashion-MNIST Dirichlet 0.1 versus 70.66% for FedCDA.
- Under noisy labels, FedBSS obtains the best results in all nine settings, e.g. 38.96% on CIFAR-10 at noise ratio 0.5 versus 30.41% for FedLC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedBSS instead of the strongest baseline | Federated learning with label distribution skew · CIFAR-10 Dirichlet α=0.5 | accuracy: 40.76 -> 45.27 | Helped |
| Use FedBSS instead of the strongest baseline | Federated learning with label distribution skew · Fashion-MNIST Shards S=8 | accuracy: 82.51 -> 79.31 | Hurt |
| Use FedBSS instead of the strongest baseline | Federated learning with label distribution skew · CIFAR-10 Shards S=8 | accuracy: 36.18 -> 34.04 | Hurt |
| Use FedBSS instead of the strongest baseline | Federated learning with noisy labels · CIFAR-10 noisy labels 0.5 | accuracy: 24.78 -> 38.96 | Helped |
| Add a 50-round diversified-knowledge warmup stage (ablation) | FedBSS · Federated learning; setting in warmup ablation figure not specified | accuracy | Helped |
| Use 100 warmup rounds instead of the 50-round warmup (ablation) | FedBSS · Federated learning; setting in warmup ablation figure not specified | accuracy | Hurt |
| Progressively add biased samples rather than filtering them out (ablation) | FedBSS · CIFAR-10, ResNet-18 | accuracy | Helped |
| Progressively add biased samples rather than linearly adding them (ablation) | FedBSS · CIFAR-10, ResNet-18 | accuracy | Helped |
Surprising findings
- Scaffold outperforms FedBSS on some datasets, and the authors note Scaffold is unstable and sensitive to datasets and hyperparameters.
- FedBSS does not achieve the best results on small datasets and the simple CNN, which the authors attribute to limited client drift on such settings.
Limitations
- Warm-up length is a sensitive hyperparameter; too short or too long hurts performance (fixed at 50 rounds).
- Gains are small or absent on small datasets and simple CNN models.
- Reported accuracy is averaged over only the final 10 rounds, with no statistical significance testing.
Experimental setup
Datasets: Fashion-MNIST CIFAR-10 CIFAR-100 DomainNet
Benchmarks: Fashion-MNIST CIFAR-10 CIFAR-100 DomainNet
Metrics: test accuracy
Baselines: FedAvg FedProx FedExP FedLC FedRS Scaffold FedCDA InCo
Builds on: FedAvg SCAFFOLD FedProx FedDyn
Closest papers by meaning
- Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning (NeurIPS 2024 · similarity 0.92)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.91)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.91)
- FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation (NeurIPS 2026 · similarity 0.91)
- Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness (AAAI 2025 · similarity 0.91)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.