Research map / Federated learning and privacy / Personalized and heterogeneous FL
Harnessing Heterogeneous Statistical Strength for Personalized Federated Learning via Hierarchical Bayesian Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hierarchical Bayesian inference framework for personalized federated learning that places a hyper-prior over personalized posterior parameters, enabling joint global and local posterior inference that shares statistical strength across clients.
Problem
Bayesian personalized federated learning aggregates personalized posteriors with heuristic rules (arithmetic means or products), which introduces strong bias and yields global models that generalize poorly under statistical heterogeneity.
Approach
Define a conjugate-style hyper-prior (Gaussian times Half-Normal) over the personalized posterior parameters so the server computes a global posterior via variational inference, with clients regularizing their personalized BNN posteriors toward it; existing Bayesian PFL methods (pFedBayes, pFedVEM) are shown to be special cases.
Key results
- On CIFAR10, global model outperforms competing methods by 5.5% and 3.7% for 50 and 100 clients (Table 1).
- On CIFAR100, personalized model surpasses competing algorithms by 5.6%, 6.0%, and 5.3% for 50, 100, and 200 clients.
- Ablation shows that regularization with (lambda1, lambda2)=(5,5) gives best CIFAR100 results, while no regularization (0,0) underperforms.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hierarchical Bayesian inference instead of pFedVEM | Ours · FMNIST, 50 clients, GM | accuracy | Helped |
| use hierarchical Bayesian inference instead of pFedVEM | Ours · FMNIST, 200 clients, GM | accuracy | Hurt |
| use hierarchical Bayesian inference instead of the strongest baseline | Ours · CIFAR10, GM, 50 and 100 clients | accuracy: +5.5%, 3.7% | Helped |
| use hierarchical Bayesian inference instead of pFedVEM | Ours · CIFAR10, 200 clients, GM | accuracy | No clear effect |
| use hierarchical Bayesian inference instead of pFedVEM | Ours · CIFAR100, GM, 50, 100, and 200 clients | accuracy: 3.2%, 3.1%, and 3.2% | Helped |
| set λ1 and λ2 to 0 instead of using (5, 5) (ablation) | Ours · CIFAR100, PM, 100 clients | accuracy: 62.2 ± 0.3 vs. 59.4 ± 0.4 | Hurt |
| set λ1 and λ2 to 0 instead of using (5, 5) (ablation) | Ours · CIFAR100, GM, 100 clients | accuracy: 55.4 ± 0.1 vs. 52.9 ± 0.2 | Hurt |
| set λ2 to 0, using (5, 0) instead of (5, 5) (ablation) | Ours · CIFAR100, GM, 200 clients | accuracy: 52.4 ± 0.4 vs. 48.1 ± 0.8 | Hurt |
| set λ1 to 0, using (0, 5) instead of (5, 5) (ablation) | Ours · CIFAR100, GM, 200 clients | accuracy: 52.4 ± 0.7 vs. 52.4 ± 0.7 | No clear effect |
| increase regularization coefficients to (10, 10) from (5, 5) (ablation) | Ours · CIFAR100, GM, 100 clients | accuracy: 55.4 ± 0.1 vs. 50.1 ± 3.7 | Hurt |
Surprising findings
- On FMNIST with 100 and 200 clients the global model is only comparable to state-of-the-art rather than clearly superior, which the authors attribute to dataset simplicity.
Limitations
- Hyperparameters for the hyper-prior regularization were grid-searched on 100 clients and transferred to 50 and 200 clients.
- Excessive regularization (10,10) degrades global model performance.
- Theoretical convergence result assumes fixed hyperparameters and a Dirac/point approximation of the global posterior.
Experimental setup
Datasets: FMNIST CIFAR10 CIFAR100 synthetic data (Li et al., 2020)
Benchmarks: FMNIST CIFAR10 CIFAR100 synthetic heterogeneity dataset
Metrics: test accuracy (%) personalized model accuracy global model accuracy
Baselines: FedAvg FedProx Scaffold FedPer FedRep PerFedAvg pFedME pFedBayes pFedVEM Local BNN only
Builds on: pFedBayes pFedVEM Bayes-by-Backprop FedAvg pFedME
Closest papers by meaning
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.91)
- Understanding the Statistical Accuracy-Communication Trade-off in Personalized Federated Learning with Minimax Guarantees (ICML 2025 · similarity 0.91)
- Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation (NeurIPS 2025 · similarity 0.90)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.89)
- Adaptive Latent-Space Constraints in Personalized Federated Learning (NeurIPS 2025 · similarity 0.89)
- Flexible Sharpness-Aware Personalized Federated Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.