Research map / Federated learning and privacy / Personalized and heterogeneous FL

Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space

AAAI 2024 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated learning with pretrained models · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes β-Predictive Bayes, a single-round Bayesian federated learning method that interpolates between the predictive mixture and product (BCM) of client predictive posteriors via a tuned β, then distills the ensemble into one model.

Problem

Bayesian federated learning methods that aggregate local predictive posteriors by product (BCM) give overconfident predictions under homogeneous data, hurting calibration in one-round FL with small, noisy, heterogeneous client datasets.

Approach

Aggregate client predictive posteriors with a log-linear interpolation between the BCM product and the predictive mixture, tune the single scalar beta on a server-side dataset by NLL, and distill the resulting ensemble into one model; theory shows BCM is overconfident for homogeneous data and mixture is underconfident for heterogeneous data.

Key results

• On regression, D-β-PredBayes achieves the lowest NLL on four of five UCI datasets (e.g., Real Estate 0.18 vs 0.49 for FedBE; Bike 0.32 vs 0.94).
• On classification, β-PredBayes gives the lowest NLL across MNIST, Fashion MNIST, EMNIST, CIFAR10 and CIFAR100 as heterogeneity increases, and lower ECE than the mixture model at h=0.9.
• Product (BCM) aggregation has the worst or near-worst NLL on regression (e.g., Air Quality 10.17 vs 7.05 for β-PredBayes), supporting the overconfidence analysis.

Measured effects reported in the paper

Surprising findings

• The distilled β-PredBayes outperforms its non-distilled ensemble on most regression datasets, possibly due to a regularization effect of the smaller model.
• The authors argue that the BCM is in fact calibrated under heterogeneous partitions, contrary to a prior proposition claiming overconfidence there, due to differing asymptotic assumptions.

Limitations

• Learns a single global model, not personalized per client when p(y|x) differs across clients.
• Transmits MCMC weight samples to server, which may leak information about client data without privacy mechanisms.
• Requires a public server-side dataset for distillation and tuning beta.

Experimental setup

Datasets: MNIST Fashion MNIST EMNIST CIFAR10 CIFAR100 Wine Quality Air Quality Forest Fire Real Estate Bike Rental

Benchmarks: MNIST Fashion MNIST EMNIST CIFAR10 CIFAR100 Wine Quality Air Quality Forest Fire Real Estate Bike Rental

Metrics: negative log-likelihood expected calibration error

Baselines: FedAvg FedProx FedPA AdaptFL FedBE OneshotFL EP-MCMC BCM Mixture model

Builds on: Bayesian Committee Machine Embarrassingly Parallel MCMC FedAvg cyclic SGHMC

Closest papers by meaning

• FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler (ICML 2024 · similarity 0.90)
• Improving Generalization in Federated Learning with Model-Data Mutual Information Regularization: A Posterior Inference Approach (NeurIPS 2024 · similarity 0.90)
• Unlocking the Potential of Model Calibration in Federated Learning (ICLR 2025 · similarity 0.89)
• FedBEns: One-Shot Federated Learning based on Bayesian Ensemble (ICML 2025 · similarity 0.88)
• Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework (ICML 2025 · similarity 0.88)
• Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation (NeurIPS 2025 · similarity 0.88)