Research map / Federated learning and privacy / Personalized and heterogeneous FL
pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes pFedClub, a personalized federated learning method that aggregates heterogeneous client models by controllable block-wise substitution (CMSR search plus stitching and similarity-based selection) and distills the selected teacher into each client.
Problem
Personalized federated learning with heterogeneous client model architectures, where existing aggregation methods lack personalization, leak privacy, need prior knowledge, or generate personalized models much larger than client capacity.
Approach
Decompose client models into blocks, group functionally similar blocks via CKA and K-means, and build candidate personalized models by order-constrained block substitution with block completion; stitch and select the candidate most similar to the client's logits as a distillation teacher, with an optional model-size constraint for controllability.
Key results
- On SVHN under Model Zoo I non-IID with labeled public data, pFedClub reaches 82.65% vs pFedHR 80.88%; on CIFAR-10 non-IID it reaches 73.62% vs pFedHR 69.88%.
- Ablation on SVHN and CIFAR-10 (Model Zoo III non-IID): removing order constraint drops SVHN accuracy from 81.03% to 61.08%.
- Controllability: average model size change for client 1 is 1.5x original for pFedClub versus 4.8x for pFedHR; pFedClub+ keeps generated models no larger than originals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pFedClub instead of pFedHR | heterogeneous federated learning with Model Zoo III and labeled public data · SVHN IID | accuracy (%): 81.88 -> 82.69 | Helped |
| use pFedClub instead of pFedHR | heterogeneous federated learning with Model Zoo III and labeled public data · CIFAR-10 non-IID | accuracy (%): 69.08 -> 71.94 | Helped |
| use pFedClub instead of pFedHR | homogeneous CNN2 federated learning · MNIST non-IID clients | accuracy (%): 92.62 -> 92.18 | Hurt |
| select the most similar first-block substitute rather than randomly selecting one (ablation) | pFedClub · SVHN IID, Model Zoo III | accuracy (%): 82.50 -> 78.69 | Hurt |
| use the smallest-index first-block substitute rather than random selection (ablation) | pFedClub · SVHN non-IID, Model Zoo III | accuracy (%): 81.03 -> 72.09 | Hurt |
| remove the order constraint from block search (ablation) | pFedClub · CIFAR-10 non-IID, Model Zoo III | accuracy (%): 71.68 -> 58.14 | Hurt |
| remove block completion after order-constrained search stops (ablation) | pFedClub · SVHN non-IID, Model Zoo III | accuracy (%): 81.03 -> 59.47 | Hurt |
| add the model-size constraint (pFedClub+) | pFedClub · SVHN non-IID | accuracy: 83.21 -> 82.51 | Hurt |
Surprising findings
- Removing the order constraint in block search causes a larger accuracy drop than simplifying anchor block selection, indicating order maintenance matters more than anchor choice.
Limitations
- CMSR is a heuristic algorithm, so generated models may be suboptimal.
- Server-side running time remains much longer than traditional averaging methods.
- Only image classification evaluated; recurrent networks not decomposed.
Experimental setup
Datasets: MNIST SVHN CIFAR-10
Benchmarks: MNIST SVHN CIFAR-10
Metrics: average client accuracy model size change percentage server running time time to target accuracy
Baselines: FedMD FedGH pFedHR FedKEMF FCCL HeteroFL FlexiFed FedAvg FedProx Per-FedAvg PFedMe PFedBayes
Builds on: pFedHR FedMD FedKEMF HeteroFL FlexiFed
Closest papers by meaning
- pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning (AAAI 2025 · similarity 0.90)
- PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers (AAAI 2025 · similarity 0.90)
- pFedMxF: Personalized Federated Class-Incremental Learning with Mixture of Frequency Aggregation (CVPR 2025 · similarity 0.89)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.89)
- Decentralized Directed Collaboration for Personalized Federated Learning (CVPR 2024 · similarity 0.89)
- pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.