Research map / Federated learning and privacy / Personalized and heterogeneous FL
Personalized Federated Learning via Feature Distribution Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes pFedFDA, which trains a shared feature extractor against a global class-conditional Gaussian (LDA-style) classifier and personalizes each client by interpolating local and global Gaussian feature statistics, improving accuracy under covariate shift and data scarcity.
Problem
Personalized federated learning under heterogeneous clients with feature distribution shift (e.g., covariate shift) and limited local data, where local classifiers suffer high variance and global classifiers high bias.
Approach
Frame representation learning as generative modeling: train the shared backbone with a global class-conditional Gaussian (tied covariance) generative classifier, then personalize each client by adapting the global Gaussian mean/covariance to local feature statistics via a learned local-global interpolation coefficient beta chosen by k-fold validation loss.
Key results
- On CIFAR10-S Dir(0.5), pFedFDA reaches 0.724 vs 0.682 for the second-best (FedAvgFT) at 100% of data and 0.595 vs 0.533 (FedPAC) at 25% of data.
- Under extreme data scarcity (one mini-batch of 50 per client), pFedFDA gets 0.499 on CIFAR100 Dir(0.5) vs 0.407 for FedPAC.
- On standard non-covariate-shift benchmarks, pFedFDA is competitive, typically within the top 3 and often within about 1% of the best method (e.g., CIFAR10 Dir(0.5) 0.763 vs FedPAC 0.767).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pFedFDA instead of the strongest baseline | Personalized federated learning · CIFAR10-S, 100%, 75%, 50%, and 25% training data | test accuracy: 4.2%; 6.9% over the second-best method | Helped |
| Use pFedFDA instead of the strongest baseline | Personalized federated learning · CIFAR100-S, 100%, 75%, 50%, and 25% training data | test accuracy: 0.1% to 6.5% | Helped |
| Use pFedFDA instead of the strongest baseline | Personalized federated learning · EMNIST Writers and CIFAR10 Dir(0.1) | test accuracy: .844(.10) vs .856(.09); .902(.09) vs .908(.09) | Mixed |
| Use pFedFDA instead of FedPAC | Personalized federated learning · CIFAR100 Dir(0.5) | test accuracy: .378(.06) -> .385(.07) | Helped |
| Use pFedFDA instead of the strongest baseline | Personalized federated learning · TinyImageNet Dir(0.1) and Dir(0.5) | test accuracy: .388(.07) vs .384(.07); .218(.04) vs .214(.04) | Hurt |
| Use pFedFDA instead of FedPAC | Personalized federated learning · CIFAR10 and CIFAR100 Dir(0.5), 50 training examples/client | test accuracy: .710(.16) -> .725(.16); .407(.15) -> .499(.16) | Helped |
| Use local-global interpolation rather than local-only feature-distribution estimates (ablation) | pFedFDA · CIFAR100 Dir(0.1) | test accuracy: .458(.08) -> .523(.08) | Helped |
| Use local-global interpolation rather than local-only feature-distribution estimates (ablation) | pFedFDA · CIFAR100-25% Dir(0.1) | test accuracy: .382(.09) -> .396(.09) | Helped |
| Use separate interpolation β terms for means and covariance rather than one shared β (ablation) | pFedFDA · CIFAR100-25% Dir(0.1) | test accuracy: .396(.09) -> .423(.09) | Helped |
| Use separate interpolation β terms for means and covariance rather than one shared β (ablation) | pFedFDA · CIFAR100 Dir(0.1) | test accuracy: .523(.08) -> .514(.08) | Hurt |
Surprising findings
- Gains from the global-local interpolation are largest in the data-scarce and covariate-shift regimes, while on moderate benchmarks the method only matches rather than clearly beats FedPAC and FedAvgFT.
Limitations
- The class-conditional Gaussian assumption may not hold for all architectures, e.g., ReLU features are better modeled by truncated Gaussians.
- Ignores cluster structure among clients, which could reduce bias in collaboration.
- Interpolation coefficient optimization accounts for most of the local overhead (9.9% to 19.4% of local training time).
Experimental setup
Datasets: EMNIST CIFAR10 CIFAR100 TinyImageNet DIGIT-5
Benchmarks: EMNIST CIFAR10 CIFAR100 TinyImageNet CIFAR10-S CIFAR100-S DIGIT-5
Metrics: average client test accuracy standard deviation of client accuracy
Baselines: Local FedAvg FedAvgFT APFL Ditto pFedMe FedRoD FedBABU FedPAC FedRep LG-FedAvg
Builds on: FedRep FedPAC FedAvg Ditto
Closest papers by meaning
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.91)
- pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning (AAAI 2025 · similarity 0.90)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.90)
- PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers (AAAI 2025 · similarity 0.89)
- Adaptive Latent-Space Constraints in Personalized Federated Learning (NeurIPS 2025 · similarity 0.89)
- Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.