Research map / Federated learning and privacy / Communication-efficient FL
FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedLPS, a federated learning framework that splits each client's model into a frozen shared pretrained encoder and per-task predictors, prunes predictors channel-wise by client capability, and aggregates heterogeneous predictors using the backbone's weights.
Problem
Federated learning on edge devices where each device must run multiple tasks at once under limited resources, data heterogeneity (non-IID) and system heterogeneity (stragglers); existing FL methods optimize one model per task.
Approach
Share a frozen pretrained encoder across tasks on a device (transfer learning) and train only task-specific predictors, with capability-dependent channel-wise L1 pruning of predictors and a heterogeneous aggregation that recovers pruned weights from the backbone predictor before weighted averaging.
Key results
- On ResNet18 with IID data, FedLPS reaches 86.27% average accuracy vs 84.31% for the best baseline (Hermes).
- With n=ceil(0.25N), FedLPS cuts FLOPs by 21.3% relative to the original multi-task setting while keeping higher accuracy than baselines (81.08% non-IID on ResNet18).
- On ShuffleNetV2 with non-IID data, FedLPS averages 77.38% vs 72.50% for Hermes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedLPS instead of Hermes | ShuffleNetV2 · IID average across five datasets | accuracy (%): 78.43 -> 81.14 | Helped |
| Use FedLPS instead of Hermes | ShuffleNetV2 · non-IID average across five datasets | accuracy (%): 72.50 -> 77.38 | Helped |
| Use FedLPS instead of FedProx | ResNet18 · IID average across five datasets | accuracy (%): 84.73 -> 86.27 | Helped |
| Use FedLPS instead of Hermes | ResNet18 · non-IID average across five datasets | accuracy (%): 79.70 -> 81.08 | Helped |
| Increase shared encoder depth from ⌈0.25N⌉ to ⌈0.5N⌉ (ablation) | FedLPS on ResNet18 · IID average | accuracy (%): 86.27 -> 82.04 | Hurt |
| Increase shared encoder depth from ⌈0.25N⌉ to ⌈0.5N⌉ (ablation) | FedLPS on ResNet18 · non-IID average | accuracy (%): 81.08 -> 77.20 | Hurt |
| Increase shared encoder depth from ⌈0.25N⌉ to ⌈0.75N⌉ (ablation) | FedLPS on ResNet18 · IID average | accuracy (%): 86.27 -> 71.12 | Hurt |
| Increase shared encoder depth from ⌈0.25N⌉ to ⌈0.75N⌉ (ablation) | FedLPS on ResNet18 · non-IID average | accuracy (%): 81.08 -> 65.57 | Hurt |
| Increase shared encoder depth from ⌈0.25N⌉ to ⌈0.5N⌉ (ablation) | FedLPS on ResNet18 · training cost | FLOPs: 135.2M -> 101.5M | Helped |
| Increase shared encoder depth from ⌈0.25N⌉ to ⌈0.75N⌉ (ablation) | FedLPS on ResNet18 · training cost | FLOPs: 135.2M -> 67.9M | Helped |
| Vary pruning ratio from 0.2 to 0.8 (ablation) | FedLPS, Hermes, and FedMP on ResNet18 · non-IID CIFAR10 and CIFAR100 | accuracy | Mixed |
Surprising findings
- Increasing the shared-encoder depth to 0.75N sharply degrades accuracy (71.12% IID on ResNet18), showing a strong accuracy-cost trade-off.
Limitations
- Encoder depth n is a fixed hyperparameter and cannot be adjusted dynamically during FL training.
- Evaluation is limited to classification tasks.
- Assumes identical model structure across all tasks; model heterogeneity is not addressed.
Experimental setup
Datasets: MNIST FashionMNIST SVHN CIFAR10 CIFAR100 ImageNet
Benchmarks: MNIST FashionMNIST SVHN CIFAR10 CIFAR100
Metrics: average accuracy (%) FLOPs communication overhead (parameters)
Baselines: FedAvg FedDrop FedProx Hermes FedMP
Builds on: FedAvg FedDrop FedProx Hermes FedMP network slimming
Closest papers by meaning
- DapperFL: Domain Adaptive Federated Learning with Model Fusion Pruning for Edge Devices (NeurIPS 2024 · similarity 0.88)
- FedCompass: Efficient Cross-Silo Federated Learning on Heterogeneous Client Devices Using a Computing Power-Aware Scheduler (ICLR 2024 · similarity 0.88)
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.88)
- Convergence Analysis of Split Federated Learning on Heterogeneous Data (NeurIPS 2024 · similarity 0.88)
- Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration (NeurIPS 2024 · similarity 0.88)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.