Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Evaluates 12 federated learning heterogeneity methods across five Dirichlet (alpha=0.1) scenarios on CIFAR-10, SVHN and CIFAR-100, and proposes FedGPS, which combines a surrogate-Gaussian distribution alignment objective with gradient-direction rectification from other clients' updates.
Problem
Existing federated learning methods for data heterogeneity show limited robustness across diverse heterogeneity scenarios (different Dirichlet partitions and client sampling settings), so no single method reliably performs best.
Approach
Static goal-oriented objective: each client aligns its local feature distribution with a privacy-free surrogate Gaussian-class distribution and then with a global surrogate, bounded via Wasserstein-1; dynamic path-oriented rectification: at each local step, the gradient is evaluated at a point shifted along the aggregated update direction of other clients from the previous round.
Key results
- On CIFAR-10 (K=10, alpha=0.1, 50% sampling), FedGPS averages 86.73 ± 3.23 mean accuracy across 5 scenarios, versus 85.69 for VHL and 75.69 for FedAvg.
- On CIFAR-10 with K=100 and 10% sampling, FedGPS averages 77.04 ± 1.19, versus 74.98 for FedLESAM and 46.69 for FedAvg.
- On CIFAR-100, FedGPS averages 70.69 ± 0.64 across 5 scenarios, above VHL's 70.16.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedGPS instead of VHL | Federated learning on CIFAR-10 with K=10 · CIFAR-10, heterogeneity scenarios 1–5 (K=10) | mean accuracy: 85.69 \pm 3.10 -> 86.73 \pm 3.23 | Helped |
| Use FedGPS instead of VHL | Federated learning on SVHN with K=10 · SVHN, heterogeneity scenarios 1–5 (K=10) | mean accuracy: 94.66 \pm 0.61 -> 95.00 \pm 0.45 | Helped |
| Use FedGPS instead of VHL | Federated learning on CIFAR-100 with K=10 · CIFAR-100, heterogeneity scenarios 1–5 (K=10) | mean accuracy: 70.16 \pm 0.91 -> 70.69 \pm 0.64 | Helped |
| Use FedGPS instead of FedLESAM | Federated learning on CIFAR-10 with K=100 · CIFAR-10, heterogeneity scenarios 1–5 (K=100) | mean accuracy: 74.98 \pm 1.88 -> 77.04 \pm 1.19 | Helped |
| Add the static goal-oriented objective (ablation) | FedAvg equipped with FedGPS modules · Diverse heterogeneous scenarios | accuracy | Helped |
| Add dynamic path-oriented gradient rectification (ablation) | FedAvg equipped with FedGPS modules · Diverse heterogeneous scenarios | accuracy | Helped |
| Combine the static objective and dynamic path rectification (ablation) | FedGPS · Diverse heterogeneous scenarios | accuracy | Helped |
| Use FedGPS-CF instead of FedGPS | FedGPS on CIFAR-10, heterogeneous scenario 1 · CIFAR-10, K=10, E=1 | accuracy: 90.31 -> 90.01 | No clear effect |
| Use FedGPS-CF instead of FedGPS | FedGPS on CIFAR-10, heterogeneous scenario 1 · CIFAR-10, K=10, E=5 | accuracy: 88.47 -> 88.13 | No clear effect |
| Use FedGPS-CF instead of FedGPS | FedGPS on CIFAR-10, heterogeneous scenario 1 · CIFAR-10, K=100, E=1 | accuracy: 78.32 -> 78.07 | No clear effect |
Surprising findings
- CCVR reaches target accuracy in fewer rounds than FedGPS in some scenarios despite lower overall accuracy.
- Most existing methods, including VHL which is best on one setting, degrade substantially under other settings, so no baseline is consistently SOTA.
Limitations
- Relies on additional surrogate data for distribution alignment, which may constrain limited-resource settings.
- Does not address heterogeneous feature distributions across clients.
- Gains on SVHN are modest because FedAvg already approximates centralized performance there.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 SVHN
Benchmarks: CIFAR-10 CIFAR-100 SVHN
Metrics: top-1 accuracy communication rounds to target accuracy speedup over FedAvg Nemenyi critical distance
Baselines: FedAvg FedAvgM FedProx SCAFFOLD CCVR VHL FedASAM FedExp FedDecorr FedDisco FedInit FedLESAM
Builds on: FedAvg SCAFFOLD FedProx CCVR VHL
Closest papers by meaning
- A New Theoretical Perspective on Data Heterogeneity in Federated Optimization (ICML 2024 · similarity 0.92)
- Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.91)
- Benchmarking Algorithms for Federated Domain Generalization (ICLR 2024 · similarity 0.90)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.90)
- Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg (ICML 2025 · similarity 0.90)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.