Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedP3, a federated framework that assigns each client a personalized server-to-client global pruning and transmits only a subset of layers back to the server, with a locally differentially private variant (LDP-FedP3) and convergence analysis.
Problem
Federated learning under client-side model heterogeneity (varying memory, compute, bandwidth), where clients need personalized smaller models and sending full models back to the server risks privacy and communication cost.
Approach
Combine personalized server-to-client global pruning with client-specific local pruning, and have each client upload only a selected subset of layers to the server, aggregated layer-wise (simple, weighted, or attention averaging), extending independent subnetwork training with a sketch-based theory and an LDP variant.
Key results
- Layer-subset training with OPU3 reaches comparable performance to FedAvg while communicating about 80% of parameters.
- On ResNet18, -B2(part) and -B3(part) cut communication by 6.25% relative to full with a 1.03% average accuracy drop; versus unpruned FedAvg the reduction is 16.63%.
- Global pruning ratio 0.9 causes an average 5.32% performance drop relative to unpruned, 12.86% at 0.7 and 27.76% at 0.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prune first or last two layers in B3 (ablation) | FedP3 with ResNet18 · CIFAR10, class-wise non-iid | accuracy | Helped |
| partially prune B2 and B3 (ablation) | FedP3 with ResNet18 · CIFAR10 and CIFAR100 | communication cost: 6.25% reduction | Helped |
| partially prune B2 and B3 (ablation) | FedP3 with ResNet18 · CIFAR10 and CIFAR100 | performance: 1.03% average decrease | Hurt |
| fully prune B2 and B3 (ablation) | FedP3 with ResNet18 · CIFAR10 and CIFAR100 | performance: 6.32% performance drop | Hurt |
| reduce global pruning ratio from 1.0 to 0.9 (ablation) | FedP3 · four datasets, two data distributions | performance: 5.32% | Hurt |
| reduce global pruning ratio from 1.0 to 0.7 (ablation) | FedP3 · four datasets, two data distributions | performance: 12.86% | Hurt |
| reduce global pruning ratio from 1.0 to 0.5 (ablation) | FedP3 · four datasets, two data distributions | performance: 27.76% | Hurt |
| replace uniform local pruning with ordered dropout (ablation) | FedP3 · CIFAR10, CIFAR100, EMNIST-L, FashionMNIST | accuracy | Hurt |
| use weighted instead of simple aggregation (ablation) | FedP3 · CIFAR10 | performance: 1.01% | Helped |
| use weighted instead of simple aggregation (ablation) | FedP3 · CIFAR100 | performance: 1.05% | Helped |
| assign clients 2-3 layers instead of 1-3 layers (ablation) | FedP3 OPU layer assignment · CIFAR10 and CIFAR100 | performance: 1.89% | Helped |
| use OPU3 layer selection (ablation) | FedP3 · CIFAR10/100 and FashionMNIST | communication cost: 20% communication reduction | Helped |
Surprising findings
- Dropping random layers from ResNet18 does not significantly hurt performance and sometimes improves it.
- Ordered Dropout is no better than simple uniform local pruning in the reported experiments.
Limitations
- Experiments are limited to small CNNs/MLPs and ResNet18 on image benchmarks; no LLM experiments despite motivation.
- Theoretical analysis assumes single-step gradient descent and uniform layer sizes, with simplifications for non-divisible dimensions.
- Reported communication savings are modest relative to unpruned FedAvg, and accuracy drops grow substantially at aggressive global pruning ratios.
Experimental setup
Datasets: CIFAR10 CIFAR100 EMNIST-L FashionMNIST
Benchmarks: CIFAR10 CIFAR100 EMNIST-L FashionMNIST
Metrics: top-1 accuracy communication cost
Baselines: FedAvg FedCR FjORD FedRolex Flado
Builds on: FedAvg Independent Subnetwork Training Ordered Dropout (FjORD) FedCR
Closest papers by meaning
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.91)
- Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning (ICML 2024 · similarity 0.90)
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.89)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.88)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.88)
- Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.