Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedGVI, a federated generalised variational inference framework that uses robust losses and weighted/alpha-Renyi divergences to be robust to prior and likelihood misspecification while recovering PVI and FedAvg as special cases.
Problem
Federated learning and Bayesian federated inference are not robust to model misspecification (contaminated labels, misspecified priors), leading to biased predictions and overconfident uncertainty.
Approach
Replaces the negative log-likelihood in partitioned variational inference with robust losses (generalised cross-entropy, density-power, gamma, score matching) and a weighted KL or alpha-Renyi divergence regularised toward the client cavity distribution, with a server-side KL update forming a logarithmic opinion pool via damping; conjugate Gaussian client updates keep cost unchanged versus PVI.
Key results
- On MNIST with 10% label flipping and 10 clients, FedGVI with AR divergence plus GCE loss reaches 97.50% accuracy versus 96.64% for FedAvg and 95.56% for PVI.
- On FashionMNIST with 40% contamination, FedGVI (delta=0.8) reaches 87.2% accuracy versus 49.0% for FedAvg and 82.8% for PVI.
- On the 2D misspecified logistic regression, FedGVI closely recovers the uncontaminated target posterior while PVI is severely impacted by outliers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedGVI with α-Rényi divergence and GCE instead of FedAvg | Mean-field Gaussian BNN · MNIST, 10% contamination, 10 clients | accuracy: 96.64$\pm$ 0.07 -> 97.50$\pm$ 0.07 | Helped |
| Use FedGVI with α-Rényi divergence and GCE instead of PVI | Mean-field Gaussian BNN · MNIST, 10% contamination, 3 clients | accuracy: 96.68$\pm$ 0.07 -> 98.13$\pm$ 0.08 | Helped |
| Add GCE to FedGVI with α-Rényi divergence (ablation) | FedGVI on MNIST · MNIST, 10% contamination, 10 clients | accuracy: 96.36$\pm$ 0.09 -> 97.50$\pm$ 0.07 | Helped |
| Add GCE to FedGVI with α-Rényi divergence (ablation) | FedGVI on MNIST · MNIST, 10% contamination, 3 clients | accuracy: 97.13 $\pm$ 0.13 -> 98.13$\pm$ 0.08 | Helped |
| Vary GCE parameter δ across 0.0, 0.4, 0.5, 0.8, and 1.0 (ablation) | FedGVI on FashionMNIST · FashionMNIST, 0% contamination | accuracy | Mixed |
| Set GCE parameter δ=0.5 instead of δ=0.0 (ablation) | FedGVI on FashionMNIST · FashionMNIST, 10% contamination | accuracy: 86.2$\pm$0.2 -> 88.6$\pm$0.2 | Helped |
| Set GCE parameter δ=0.5 instead of δ=0.0 (ablation) | FedGVI on FashionMNIST · FashionMNIST, 20% contamination | accuracy: 85.6$\pm$0.1 -> 88.4$\pm$0.2 | Helped |
| Set GCE parameter δ=0.8 instead of δ=0.0 (ablation) | FedGVI on FashionMNIST · FashionMNIST, 40% contamination | accuracy: 83.8$\pm$0.1 -> 87.2$\pm$0.1 | Helped |
| Set GCE parameter δ=0.4 rather than δ=0.8 (ablation) | FedGVI on FashionMNIST · FashionMNIST, 40% contamination | accuracy: 87.2$\pm$0.1 -> 78.1$\pm$0.4 | Hurt |
Surprising findings
- Robust FedGVI with GCE at delta=0.0 (the NLL case) underperforms several non-robust baselines such as FedPA and beta-PredBayes on FashionMNIST at 0% contamination, while higher delta does better.
- The FedGVI ablation indicates that a single fixed hyperparameter setting (e.g., delta 0.4-1.0) performs well across contamination levels, though delta=0.4 drops to 78.1% at 40% contamination.
Limitations
- Robustness to prior misspecification is not provably established and is left as an open problem.
- Robust losses may underperform when the likelihood is correctly specified.
- The global GVI limit for non-KL server divergences is an open problem, so theory restricts the server divergence to KL.
Experimental setup
Datasets: MNIST FashionMNIST Covertype synthetic 1D clutter synthetic 2D logistic regression
Benchmarks: MNIST FashionMNIST Covertype 1D clutter problem 2D misspecified logistic regression
Metrics: classification accuracy negative log likelihood Fisher-Rao distance (influence function)
Baselines: FedAvg PVI FedPA DSVGD DSGLD beta-PredBayes MLE VI GVI
Builds on: Partitioned Variational Inference Generalised Variational Inference Generalised Bayesian Inference Federated Averaging Expectation Propagation
Closest papers by meaning
- Improving Generalization in Federated Learning with Model-Data Mutual Information Regularization: A Posterior Inference Approach (NeurIPS 2024 · similarity 0.89)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.88)
- Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation (NeurIPS 2025 · similarity 0.88)
- Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Silos (ICML 2025 · similarity 0.88)
- Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space (AAAI 2024 · similarity 0.88)
- Connecting Federated ADMM to Bayes (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.