Research map / Federated learning and privacy / Personalized and heterogeneous FL
PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PFedCS, a personalized federated learning method that selects collaborators via classifier-parameter distances (GMM-based adaptive clustering) and builds distance-constrained customized classifiers that guide local training.
Problem
Personalized federated learning under label-skewed Non-IID data, where global models underperform and existing methods either aggregate indiscriminately, rely on architecture design, or lack fine-grained classifier-level collaboration among similar clients without knowing the number of groups.
Approach
Measures pairwise distances between clients' classifier parameters each round, uses a two-component GMM per client (client as cluster center) to adaptively pick similar collaborators, aggregates a customized classifier with distance-constrained weights, and uses it as a teacher via KL distillation to guide each client's local classifier.
Key results
- Under pathological Non-IID, PFedCS reaches 61.64% average accuracy across six CIFAR/Tiny-ImageNet settings, outperforming the best baseline by an average of 2.51%.
- Under Dirichlet practical Non-IID, PFedCS achieves 68.41% average, best in all but one setting (CIFAR-10 alpha=0.05, where FedAMP is slightly better), and beats the sub-optimal method by 2.50% on Tiny-ImageNet.
- Ablation: adaptive selection improves over Select None (FedPer-equivalent) by about 1.74% on average on CIFAR-100; replacing GMM with K-Means or hierarchical clustering changes results little, while a fixed threshold drops Tiny-ImageNet accuracy by 6.70%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PFedCS instead of the strongest reported baseline | PFedCS · CIFAR-10, pathological Non-IID, C = 2 | top-1 accuracy: 90.48 -> 90.60 | Helped |
| Use PFedCS instead of the strongest reported baseline | PFedCS · CIFAR-100, pathological Non-IID, C = 20 | top-1 accuracy: 47.85 -> 50.00 | Helped |
| Use PFedCS instead of the strongest reported baseline | PFedCS · Tiny-ImageNet, pathological Non-IID, C = 30 | top-1 accuracy: 37.36 -> 41.36 | Helped |
| Use PFedCS instead of the strongest reported baseline | PFedCS · CIFAR-10, practical Non-IID, α = 0.05 | top-1 accuracy: 93.76 -> 93.58 | Hurt |
| Use adaptive collaborator selection based on classifier distances instead of selecting none (ablation) | PFedCS · CIFAR-100, pathological Non-IID | top-1 accuracy | Helped |
| Select all other clients as collaborators rather than adaptively selecting similar clients (ablation) | PFedCS · CIFAR-100, pathological Non-IID | top-1 accuracy | Hurt |
| Randomly select 3 collaborators rather than adaptively selecting similar clients (ablation) | PFedCS · CIFAR-100, pathological Non-IID | top-1 accuracy | Hurt |
| Replace GMM with K-Means for grouping clients (ablation) | PFedCS · CIFAR-10, CIFAR-100, and Tiny-ImageNet, pathological Non-IID | top-1 accuracy | No clear effect |
| Replace GMM with hierarchical clustering for grouping clients (ablation) | PFedCS · CIFAR-10, CIFAR-100, and Tiny-ImageNet, pathological Non-IID | top-1 accuracy | No clear effect |
| Replace GMM clustering with a fixed threshold of 0.5 (ablation) | PFedCS · Tiny-ImageNet, pathological Non-IID | top-1 accuracy: 46.93 to 40.23 | Hurt |
| Increase customized-classifier fine-tuning from ρ = 0 to ρ = 1 (ablation) | PFedCS · CIFAR-100, practical Non-IID, α = 0.05 | top-1 accuracy: 53.58 -> 59.39 | Helped |
| Increase customized-classifier fine-tuning from ρ = 1 to ρ = 5 (ablation) | PFedCS · CIFAR-100, practical Non-IID, α = 0.05 | top-1 accuracy: 59.39, 59.51, 59.41, 59.46 | No clear effect |
Surprising findings
- A fixed distance threshold for collaborator selection performs notably worse than clustering-based selection, showing the choice of grouping matters more than the clustering algorithm itself.
- FedAMP outperforms PFedCS on CIFAR-10 with alpha=0.05, which the authors attribute to the simpler dataset.
Limitations
- Evaluated only on label-distribution shift in Non-IID settings with small CNN/ResNet-18 models on three image datasets.
- Two-stage design degrades to FedPer after round beta, and the method relies on the classifier's parameter distance being informative, which is only argued empirically early in training.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: mean top-1 test accuracy over clients
Baselines: FedAvg FedProx Per-FedAvg FedPer FedBABU FedAMP FedFomo FedProto FedRod FedGH ClusterFL FeSEM
Builds on: FedPer FedRod FedAMP FedBABU FedAvg
Closest papers by meaning
- pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning (NeurIPS 2024 · similarity 0.90)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.90)
- PeFLL: Personalized Federated Learning by Learning to Learn (ICLR 2024 · similarity 0.89)
- Personalized Federated Learning via Feature Distribution Adaptation (NeurIPS 2024 · similarity 0.89)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.89)
- pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.