Research map / Federated learning and privacy / Personalized and heterogeneous FL
A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedBNN, a personalized Bayesian federated learning framework where clients train Bayesian neural networks of varying size and collaborate by sharing outputs on a public alignment dataset that are used to set functional-space priors, with a zCDP-based differential privacy guarantee.
Problem
Federated learning under data scarcity and heterogeneous client compute and non-IID data, where clients need reliable uncertainty-aware local models without high communication and computation costs of Bayesian FL and without leaking private data.
Approach
Clients share their model outputs on a shared unlabeled alignment dataset rather than weight distributions; the server aggregates these outputs, and each client sets its BNN weight-space prior by fitting the prior parameters so the network matches the (convex-combined) global output on that dataset, then performs Bayes-by-Backprop locally. This allows models of different architectures to collabora
Key results
- Under non-IID CIFAR-10 small setting, Ours (Homo) reaches 70.6% vs. 60.01% for the best baseline FedAUX (and 58.7% for pFedBayes).
- Heterogeneous-architecture FedBNN reaches 78.7% on CIFAR-10 full data vs. 79.19% for FedProx-type baselines' best and 95.9% on MNIST full, with about 6% average gain claimed for small and medium settings.
- With (ε≈9.98, δ=1e-4) differential privacy, Ours (Hetero-DP) keeps 89.82% on MNIST small, above DP-FedAvg's 80.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use homogeneous FedBNN instead of the strongest baseline | FedBNN · MNIST non-IID small | test accuracy: 94.0 ± 0.2 -> 94.9 ± 1.0 | Helped |
| Use homogeneous FedBNN instead of the strongest baseline | FedBNN · CIFAR-10 non-IID small | test accuracy: 60.01 ± 1.96 -> 70.6 ± 1.1 | Helped |
| Use homogeneous FedBNN instead of the strongest baseline | FedBNN · CIFAR-100 non-IID small | test accuracy: 47.06 ± 1.3 -> 49.65 ± 1.4 | Helped |
| Use heterogeneous client architectures instead of homogeneous FedBNN | FedBNN · CIFAR-10 non-IID small | test accuracy: 70.6 ± 1.1 -> 68.17 ± 2.0 | Hurt |
| Use heterogeneous client architectures instead of homogeneous FedBNN | FedBNN · CIFAR-100 non-IID full | test accuracy: 57.3 ± 0.8 -> 51.1 ± 0.7 | Hurt |
| Use CIFAR-10 AD containing all 10 classes rather than 8 classes (ablation) | FedBNN homogeneous · CIFAR-10 non-IID, medium data | test accuracy: 69.7 ± 0.28 -> 72.7 ± 0.15 | Helped |
| Use CIFAR-10 AD containing all 10 classes rather than 5 classes (ablation) | FedBNN homogeneous · CIFAR-10 non-IID, medium data | test accuracy: 68.8 ± 0.97 -> 72.7 ± 0.15 | Helped |
| Use CIFAR-10 AD containing all 10 classes rather than 2 classes (ablation) | FedBNN homogeneous · CIFAR-10 non-IID, medium data | test accuracy: 67.2 ± 1.5 -> 72.7 ± 0.15 | Helped |
| Use CIFAR-10 AD containing all 10 classes rather than SVHN (ablation) | FedBNN homogeneous · CIFAR-10 non-IID, medium data | test accuracy: 70.1 ± 0.18 -> 72.7 ± 0.15 | Helped |
| Use CIFAR-10 AD containing all 10 classes rather than 8 classes (ablation) | FedBNN heterogeneous · CIFAR-10 non-IID, medium data | test accuracy: 68.4 ± 0.80 -> 71.6 ± 0.93 | Helped |
| Use CIFAR-10 AD containing all 10 classes rather than 5 classes (ablation) | FedBNN heterogeneous · CIFAR-10 non-IID, medium data | test accuracy: 68.8 ± 1.4 -> 71.6 ± 0.93 | Helped |
| Use CIFAR-10 AD containing all 10 classes rather than 2 classes (ablation) | FedBNN heterogeneous · CIFAR-10 non-IID, medium data | test accuracy: 68.1 ± 1.9 -> 71.6 ± 0.93 | Helped |
Surprising findings
- Baseline methods' performance did not change when their encoders were pre-trained on the unlabeled alignment dataset, because their FL procedure overwrites prior local knowledge.
- An out-of-domain alignment dataset (SVHN) outperformed a 2-class in-domain CIFAR-10 alignment set.
Limitations
- Exact Bayesian posterior inference is infeasible, so variational approximation is used.
- Requires a small unlabeled public alignment dataset at the server.
- Bayesian methods are more computationally expensive than point-estimate counterparts.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100 SVHN NotMNIST
Benchmarks: MNIST CIFAR-10 CIFAR-100
Metrics: test accuracy ECE MCE entropy on out-of-distribution data
Baselines: FedAvg FedProx pFedME FedAUX non-Bayesian KD pFedGP pFedBayes FOLA DP-FedAvg
Builds on: Bayes by Backprop FedAvg FedAUX pFedBayes zCDP
Closest papers by meaning
- Harnessing Heterogeneous Statistical Strength for Personalized Federated Learning via Hierarchical Bayesian Inference (ICML 2025 · similarity 0.91)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.91)
- Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning (ICML 2024 · similarity 0.90)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.89)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.89)
- Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.