Research map / Federated learning and privacy / Personalized and heterogeneous FL
PeFLL: Personalized Federated Learning by Learning to Learn
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PeFLL, a personalized federated learning method where a server-side hypernetwork maps client descriptors (from a jointly trained embedding network) to personalized models in one forward pass, with convergence and PAC-Bayesian generalization guarantees.
Problem
Personalized federated learning must produce accurate models for clients seen during training and for future unseen clients, especially with little data, without costly on-client finetuning or multi-round communication.
Approach
Jointly train an embedding network (average of per-sample embeddings giving a client descriptor) and a server-side hypernetwork that outputs client models; the client computes its descriptor locally and the server generates the model in a single forward pass, with gradient flow split across server and client so no second-order gradients or hypernetwork transmission are needed.
Key results
- On unseen clients, PeFLL reaches 89.3% on CIFAR10 (n=90) and 38.4% on CIFAR100 (n=900), versus 82.4% and 33.7% for the best baseline kNN-Per.
- On FEMNIST, PeFLL achieves 90.1% accuracy for training clients and 90.7% for unseen clients, above the best baseline pFedMe at 85.9% (training) and 86.1% (unseen).
- Using only unlabeled data for test clients, PeFLL improves over FedAvg in all settings (e.g. 84.5% vs 48.6% on CIFAR10 with n=90).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PeFLL instead of the strongest listed baseline | Personalized federated learning · CIFAR10 and CIFAR100, seen and unseen clients; FEMNIST, seen and unseen clients | accuracy | Mixed |
| Generate personalized models with PeFLL rather than FedAvg | Unlabeled-only test clients · CIFAR10, CIFAR100, FEMNIST | accuracy | Helped |
| Add label masking (ablation) | PeFLL · CIFAR100, seen and unseen clients | accuracy: 56.0 ± 0.34 -> 59.46 ± 0.74; 43.2 ± 0.79 -> 53.23 ± 0.62; 40.9 ± 0.60 -> 51.38 ± 0.25; 35.17 ± 1.43 -> 55.60 ± 1.63; 38. | Helped |
| Add label masking (ablation) | PeFLL · CIFAR10, seen and unseen clients | accuracy: 88.96 ± 0.84 -> 88.90 ± 0.69; 88.88 ± 0.38 -> 88.94 ± 0.31; 87.78 ± 0.41 -> 88.42 ± 0.37; 89.30 ± 2.16 -> 88.90 ± 2.40; | Mixed |
| Use a larger rather than a medium hypernetwork (ablation) | PeFLL · CIFAR100, 100/500/1000 clients | accuracy: 59.4 ± 0.2 -> 59.8 ± 0.1; 53.1 ± 0.4 -> 53.8 ± 0.2; 50.8 ± 0.5 -> 52.6 ± 1.0 | Helped |
| Use a small rather than a medium hypernetwork (ablation) | PeFLL · CIFAR100, 100/500/1000 clients | accuracy: 59.4 ± 0.2 -> 57.9 ± 0.3; 53.1 ± 0.4 -> 51.9 ± 0.6; 50.8 ± 0.5 -> 47.4 ± 1.4 | Hurt |
| Use a small rather than a medium hypernetwork (ablation) | PeFLL · CIFAR10, 100/500/1000 clients | accuracy: 88.9 ± 0.7 -> 88.7 ± 0.3; 88.9 ± 0.3 -> 88.2 ± 0.4; 88.1 ± 0.3 -> 87.7 ± 0.4 | Mixed |
| Use a CNN rather than an MLP embedding network (ablation) | PeFLL · CIFAR10 and CIFAR100, 100/500/1000 clients | accuracy: 88.5 ± 1.2 -> 88.9 ± 0.7; 87.8 ± 1.1 -> 88.9 ± 0.3; 87.5 ± 2.1 -> 88.4 ± 0.4; 59.1 ± 0.2 -> 59.4 ± 0.2; 53.4 ± 0.4 -> 54 | Helped |
| Increase hypernetwork and embedding regularization from 0 to 10^-3 while client regularization is 0 (ablation) | PeFLL · CIFAR10, 100 clients | accuracy: 87.4 ± 1.1 -> 88.4 ± 0.6 | Helped |
| Increase client regularization from 0 to 5·10^-5 while hypernetwork and embedding regularization are 0 (ablation) | PeFLL · CIFAR10, 100 clients | accuracy: 87.4 ± 1.1 -> 88.7 ± 0.7 | Helped |
| Set client regularization to 5·10^-1 instead of 5·10^-5, with hypernetwork and embedding regularization 10^-3 (ablation) | PeFLL · CIFAR10, 100 clients | accuracy: 88.8 ± 0.3 -> 86.2 ± 0.4 | Hurt |
Surprising findings
- Per-round communication cost is about twice that of FedAvg, since the embedding network and client models are both sent.
- pFedHN's accuracy drops noticeably as the number of clients grows and per-client data shrinks, while PeFLL stays stable.
Limitations
- Communication per round is roughly twice that of FedAvg.
- Differential privacy or formal privacy guarantees are not provided and left to future work.
- Hypernetwork output layer is at least 100x the size of client models, making the server-side model large.
Experimental setup
Datasets: CIFAR10 CIFAR100 FEMNIST Shakespeare
Benchmarks: CIFAR10 CIFAR100 FEMNIST
Metrics: test accuracy Spearman rank correlation
Baselines: Local FedAvg Per-FedAvg FedRep pFedMe kNN-Per pFedHN
Builds on: pFedHN FedAvg FedRep Per-FedAvg pFedMe
Closest papers by meaning
- Personalized Federated Learning using Hypernetworks (ICML 2021 · similarity 0.91)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.91)
- PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers (AAAI 2025 · similarity 0.89)
- pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning (NeurIPS 2024 · similarity 0.89)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.89)
- FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuning (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.