Research map / Federated learning and privacy / Personalized and heterogeneous FL
pFedGPA: Diffusion-based Generative Parameter Aggregation for Personalized Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes pFedGPA, which trains a latent diffusion model on the server over clients' uploaded parameters and uses a parameter inversion method to produce personalized parameters for each client in personalized federated learning.
Problem
Linear parameter aggregation (e.g., FedAvg) under non-IID client data can land in low-probability regions of parameter space, degrading the aggregated model; personalized FL methods still rely heavily on linear aggregation.
Approach
Deploy a latent diffusion model on the server that learns the distribution of clients' model parameters; invert each uploaded parameter vector into a latent code (final noise plus per-step noises) and resample with deterministic noises to obtain personalized parameters, with classifier-style global guidance for newly joining clients.
Key results
- On CIFAR-10 with 100 clients, pFedGPA reaches 78.33% vs 77.12% for pFedHN (best baseline in that column), roughly 2% gain as stated.
- On EMNIST with 100 clients, pFedGPA reaches 88.22% vs 86.52% for FedAvg-FT.
- Parameter inversion raises EMNIST after-FT accuracy from 81.09% to 84.70% and Fashion-MNIST after-FT accuracy from 87.24% to 89.37%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · EMNIST, 10 clients | accuracy: 81.91 -> 82.53 | Helped |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · EMNIST, 20 clients | accuracy: 85.09 -> 84.70 | Hurt |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · EMNIST, 100 clients | accuracy: 86.52 -> 88.22 | Helped |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · Fashion-MNIST, 10 clients | accuracy: 88.45 -> 88.90 | Helped |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · Fashion-MNIST, 20 clients | accuracy: 89.39 -> 89.18 | Hurt |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · Fashion-MNIST, 100 clients | accuracy: 89.43 -> 90.08 | Helped |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · CIFAR-10, 10 clients | accuracy: 71.76 -> 72.95 | Helped |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · CIFAR-10, 20 clients | accuracy: 76.46 -> 78.54 | Helped |
| use diffusion-based generative parameter aggregation instead of the strongest listed baseline | pFedGPA · CIFAR-10, 100 clients | accuracy: 77.12 -> 78.33 | Helped |
| add parameter inversion rather than direct generation (ablation) | pFedGPA · EMNIST, 20 clients, before FT | accuracy: 69.38 -> 83.54 | Helped |
| add parameter inversion rather than direct generation (ablation) | pFedGPA · EMNIST, 20 clients, after FT | accuracy: 81.09 -> 84.70 | Helped |
| add parameter inversion rather than direct generation (ablation) | pFedGPA · Fashion-MNIST, 20 clients, before FT | accuracy: 68.48 -> 83.82 | Helped |
Surprising findings
- Direct generation without inversion was unstable, with about 3 of 20 cases giving pre-fine-tuning accuracy below 60%.
- On CIFAR-10 the inversion variant had slightly lower after-FT accuracy (75.72%) than without inversion (74.77%) only in the before-FT case; before-FT inversion was higher (76.28%) but after-FT was not, a mixed result.
Limitations
- Diffusion model training time exceeds inter-round time, creating a bottleneck.
- Only small CNNs are used; for larger models only the final fully connected layers are generated due to hardware constraints.
- Evaluated only on three image classification benchmarks with small local datasets (600 samples per client).
Experimental setup
Datasets: Fashion-MNIST EMNIST CIFAR-10
Benchmarks: Fashion-MNIST EMNIST CIFAR-10
Metrics: average test accuracy across clients
Baselines: Local-only FedAvg FedAvg-FT FedPer FedRep FedBABU LG-FedAvg pFedHN
Builds on: p-diff latent diffusion model DDPM pFedHN FedAvg
Closest papers by meaning
- pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning (NeurIPS 2024 · similarity 0.89)
- Personalized Federated Learning via Feature Distribution Adaptation (NeurIPS 2024 · similarity 0.88)
- Decentralized Directed Collaboration for Personalized Federated Learning (CVPR 2024 · similarity 0.87)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.87)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.87)
- PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.