Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedLPA, a data-free one-shot federated learning method that aggregates local models via layer-wise Laplace approximation (block-diagonal Kronecker-factored empirical Fisher) and trains global parameters by minimizing a quadratic objective on the server.
Problem
One-shot federated learning performs poorly under high statistical heterogeneity (non-IID label skew), and existing one-shot methods need auxiliary public data, pretrained models, or expose label information.
Approach
Infers each layer's local posterior via a layer-wise Kronecker-factored empirical Fisher (block-diagonal, capturing intra-layer parameter correlations), then aggregates posteriors by solving a quadratic objective over the global weight matrix with gradient descent, avoiding explicit covariance inversion and the expectation approximation of Kronecker products.
Key results
- On FMNIST with 10 clients, FedLPA reaches 21.20% at Dir(0.01) versus 18.17% for FedAvg and 15.23% for DENSE, and 13.20% vs ~10% for baselines under #C=1 (extreme skew).
- On MNIST Dir(0.1), FedLPA achieves 77.43% vs 70.33% for DENSE and 48.63% for FedAvg.
- With an MLP and different initializations, FedLPA attains 73.73% while DiagonalFisher drops to 10.51% on FMNIST.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedLPA layer-wise posterior aggregation instead of DENSE | One-shot federated learning with a 5-layer CNN · FMNIST, Dirichlet β=0.01 | accuracy: 21.20$\pm$0.67 vs 15.23$\pm$0.14 | Helped |
| Use FedLPA layer-wise posterior aggregation instead of DENSE | One-shot federated learning with a 5-layer CNN · CIFAR-10, Dirichlet β=0.01 | accuracy: 16.17$\pm$0.00 vs 12.30$\pm$0.03 | Helped |
| Use FedLPA layer-wise posterior aggregation instead of DENSE | One-shot federated learning with a 5-layer CNN · MNIST, Dirichlet β=0.01 | accuracy: 39.17$\pm$1.16 vs 15.80$\pm$0.24 | Helped |
| Use FedLPA layer-wise posterior aggregation instead of DENSE | One-shot federated learning with a 5-layer CNN · SVHN, Dirichlet β=0.01 | accuracy: 19.20$\pm$0.00 vs 17.33$\pm$0.28 | Helped |
| Use layer-wise block-diagonal Fisher aggregation instead of DiagonalFisher | MLP one-shot federated learning · FMNIST, β=0.01, same client initialization | global test accuracy: 56.34$\pm$0.34 -> 76.63$\pm$0.04 | Helped |
| Use layer-wise block-diagonal Fisher aggregation instead of DiagonalFisher | MLP one-shot federated learning · FMNIST, β=0.01, different client initialization | global test accuracy: 10.51$\pm$0.11 -> 73.73$\pm$0.07 | Helped |
| Set λ to 0.01 rather than the default 0.001 (ablation) | FedLPA · FMNIST, β=0.01 | accuracy: 21.20$\pm$0.67 -> 18.63$\pm$0.78 | Hurt |
| Set λ to 0.0001 rather than the default 0.001 (ablation) | FedLPA · FMNIST, β=0.01 | accuracy: 21.20$\pm$0.67 -> 22.50$\pm$1.84 | Helped |
| Set λ to 0.01 rather than the default 0.001 (ablation) | FedLPA · FMNIST, β=0.1 | accuracy: 55.33$\pm$0.06 -> 56.83$\pm$0.19 | Helped |
| Set λ to 0.0001 rather than the default 0.001 (ablation) | FedLPA · FMNIST, β=0.1 | accuracy: 55.33$\pm$0.06 -> 54.60$\pm$0.15 | Hurt |
| Vary λ among 0.01, 0.001, and 0.0001 (ablation) | FedLPA · FMNIST, multiple label-skew partitions | accuracy | Mixed |
Surprising findings
- DiagonalFisher aggregation completely fails (10.51%) when client models start from different initializations, while the Kronecker-factored method keeps 73.73%.
- DENSE slightly outperforms FedLPA in one setting (20 clients, Dir(0.5) on FMNIST, 74.10 vs 74.07).
Limitations
- Communication cost is higher than DiagonalFisher (about 2.27 MB vs 1.66 MB per client for the MLP); the paper notes FedLPA is slower than FedAvg-family baselines.
- Evaluated only on small image benchmarks with a 5-layer CNN and an MLP; extension to models with very large layers relies on unverified low-rank approximation discussed only in the appendix.
- Privacy analysis is partial: iDLG results show mixed evidence, and the paper leaves integration with other privacy techniques as open.
Experimental setup
Datasets: MNIST Fashion-MNIST CIFAR-10 SVHN
Benchmarks: MNIST FMNIST CIFAR-10 SVHN
Metrics: test accuracy communication overhead (MB) computation time (mins) MSE under iDLG attack
Baselines: FedAvg FedProx FedNova SCAFFOLD DENSE DiagonalFisher
Builds on: FedAvg Laplace approximation K-FAC DENSE FedPA
Closest papers by meaning
- One-shot Federated Learning via Synthetic Distiller-Distillate Communication (NeurIPS 2024 · similarity 0.89)
- FedBEns: One-Shot Federated Learning based on Bayesian Ensemble (ICML 2025 · similarity 0.88)
- Capture Global Feature Statistics for One-Shot Federated Learning (AAAI 2025 · similarity 0.88)
- One-Shot Heterogeneous Federated Learning with Local Model-Guided Diffusion Models (ICML 2025 · similarity 0.88)
- FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinking (ICLR 2025 · similarity 0.87)
- FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.