Research map / Federated learning and privacy / Personalized and heterogeneous FL
Bi-level Personalization for Federated Foundation Models: A Task-vector Aggregation Approach
Yiyuan Yang, Guodong Long, Qinghua Lu, Liming Zhu, Jing Jiang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add task-vector-similarity personalized aggregation (FedBip) over uniform FedIT aggregation | LLaMA-7B LoRA federated fine-tuning · Flan 8-task average | average score: 79.05 -> 81.21 | Improved |
| add layer-wise task-vector aggregation (FedBip-L) | FedBip · Flan 8-task average | average score: 81.21 -> 81.90 | Improved |
| add layer-wise task-vector aggregation (FedBip-L) | FedBip · OfficeHome average | accuracy: 93.41 -> 93.74 | Improved |
| plug FedBip into FedProx | FedProx · OfficeHome average | accuracy: 89.68 -> 94.44 | Improved |
| plug FedBip into FFA-LoRA | FFA-LoRA · Flan 8-task average | average score: 67.78 -> 73.77 | Improved |
| plug FedBip into FedDPA | FedDPA · Flan 8-task average | average score: 81.74 -> 82.01 | Improved |
| plug FedBip into Ditto | Ditto · OfficeHome average | accuracy: 92.39 -> 93.63 | Improved |
| use cosine similarity instead of L2 for task-vector weights (ablation) | FedBip · Flan 8-task average | average score: 79.55 -> 81.21 | Improved |
| use cosine similarity instead of Pearson for task-vector weights (ablation) | FedBip · Flan 8-task average | average score: 80.77 -> 81.21 | Improved |
| compute weights from task vectors instead of parameters, and aggregate task vectors (ablation) | FedBip · Flan 8-task average | average score: 79.63 -> 81.21 | Improved |
| aggregate task vectors rather than parameters when weights come from parameters (ablation) | FedBip · Flan 8-task average | average score: 79.63 -> 80.15 | Improved |
| scale clients from 8 to 40 with FedBip (ablation) | FedBip · Flan NLP, 40 clients | average score: 71.53 -> 73.00 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 3 of 3 on most shared results.
- FedALA — FedALA: Adaptive Local Aggregation for Personalized Federated Learning (AAAI 2023) · better on 4 of 5 shared results
- FedAWA — FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025) · better on 2 of 2 shared results
- Ditto — Ditto: Fair and Robust Federated Learning Through Personalization (ICML 2021) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Dual-Personalizing Adapter for Federated Foundation Models (NeurIPS 2024 · similarity 0.90)
- Layer-Wised Model Aggregation for Personalized Federated Learning (CVPR 2022 · similarity 0.90)
- pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning (NeurIPS 2024 · similarity 0.88)
- Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients (NeurIPS 2024 · similarity 0.87)
- FedAS: Bridging Inconsistency in Personalized Federated Learning (CVPR 2024 · similarity 0.87)
- FedCALM: Conflict-aware Layer-wise Mitigation for Selective Aggregation in Deeper Personalized Federated Learning (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
