Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedImpro: Measuring and Improving Client Update in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedImpro, which decouples the network into low- and high-level parts and trains the high-level classifier on shared, estimated Gaussian feature distributions to reduce client drift in federated learning.
Problem
Federated learning under non-IID data suffers from client drift and a generalization gap to centralized training; existing fixes mainly correct gradients and updates.
Approach
Theoretically bounds the generalization contribution of local training by a conditional Wasserstein distance between clients, then builds similar conditional feature distributions by sharing Gaussian estimates of low-level features (with noise) and training the high-level layers on sampled shared features, which reduces high-level gradient dissimilarity.
Key results
- On CIFAR-10 with alpha=0.1, E=1, M=10, FedImpro reaches 88.45% vs 83.65% for FedAvg and 84.97% for FedNova.
- On SVHN with alpha=0.05, FedImpro reaches 90.25%, about 6.14 points above FedNova.
- FedImpro reduces communication rounds to target accuracy in most settings, e.g., 32 vs 83 for FedAvg on FMNIST (alpha=0.1, E=1, M=10).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add shared reconstructed feature distributions for high-level local training | FedAvg-based federated learning · CIFAR-10 (average across FL settings) | test accuracy: 80.96 -> 83.97 | Helped |
| Add shared reconstructed feature distributions for high-level local training | FedAvg-based federated learning · FMNIST (average across FL settings) | test accuracy: 87.85 -> 89.53 | Helped |
| Add shared reconstructed feature distributions for high-level local training | FedAvg-based federated learning · SVHN (average across FL settings) | test accuracy: 89.42 -> 91.91 | Helped |
| Add shared reconstructed feature distributions for high-level local training | FedAvg-based federated learning · CIFAR-100 (average across FL settings) | test accuracy: 64.46 -> 65.44 | Helped |
| Split the model at the 5th layer instead of the standard split (ablation) | FedImpro · CIFAR-10 | test accuracy | No clear effect |
| Split the model at the 9th layer (ablation) | FedImpro · CIFAR-10 | test accuracy | Helped |
| Split the model at the 13th layer (ablation) | FedImpro · CIFAR-10 | test accuracy | Helped |
| Split the model at the 17th layer (ablation) | FedImpro · CIFAR-10 | test accuracy | Hurt |
Surprising findings
- On CIFAR-100, FedImpro performs similarly to FedProx at alpha=0.1, which the authors attribute to the larger number of classes and the simple Gaussian feature estimate.
- Decoupling at the 5th layer does not yield gains despite theory predicting more gradient-dissimilarity reduction, which the authors attribute to difficulty of estimating low-level feature distributions.
Limitations
- Gaussian approximation of feature distributions is inexact, especially on CIFAR-100 with many classes.
- Privacy protection is only shown through model inversion attack experiments; trade-off with privacy is not fully investigated.
- Theory assumes conditional distributions and only covers the high-level gradient dissimilarity; low-level part is not directly addressed.
Experimental setup
Datasets: CIFAR-10 FMNIST SVHN CIFAR-100
Benchmarks: CIFAR-10 FMNIST SVHN CIFAR-100
Metrics: best test accuracy communication rounds to target accuracy weight divergence
Baselines: FedAvg FedProx SCAFFOLD FedNova FedDyn
Builds on: FedAvg FedProx SCAFFOLD FedNova FedDyn
Closest papers by meaning
- Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.91)
- FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation (NeurIPS 2026 · similarity 0.90)
- FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model Updates (ICML 2025 · similarity 0.90)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.90)
- Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness (AAAI 2025 · similarity 0.90)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.