Research map / Federated learning and privacy / Personalized and heterogeneous FL
Heterogeneous Federated Fine-Tuning with Parallel One-Rank Adaptation
Zikai Zhang, Rui Hu, Jiahao Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace random re-initialization (FLoRA) with Select-N-Fold PLoRA | FLoRA · Natural Instructions (IID) | Rouge-L: 33.82 -> 64.96 | Improved |
| replace SVD truncation (FlexLoRA) with PLoRA Select-N-Fold | FlexLoRA · Natural Instructions (IID) | Rouge-L: 59.07 -> 64.96 | Improved |
| replace zero-padding/truncation (HETLoRA) with PLoRA Select-N-Fold | HETLoRA · Natural Instructions (IID) | Rouge-L: 58.07 -> 64.96 | Improved |
| heterogeneous Fed-PLoRA vs homogeneous FedIT rank R/2 | FedIT (R/2) · Natural Instructions (IID) | Rouge-L: 64.52 -> 64.96 | No clear effect |
| Fed-PLoRA vs FLoRA on GLUE | FLoRA · GLUE (IID) average | score: 47.81 -> 79.89 | Improved |
| Fed-PLoRA vs HETLoRA on GLUE | HETLoRA · GLUE (IID) average | score: 73.55 -> 79.89 | Improved |
| FlexLoRA vs Fed-PLoRA on QNLI | Fed-PLoRA · GLUE QNLI | score: 88.54 vs FlexLoRA 90.01 | Hurt |
| Fed-PLoRA vs FlexLoRA on financial tasks | FlexLoRA · FPB/FIQA/TFNS average | accuracy: 52.38 -> 53.27 | Improved |
| random module selection (Select-N-Drop, Select-N-Fold) instead of deterministic selection (ablation) | Fed-PLoRA · CoLA (IID) | accuracy | Improved |
| fold unselected modules into frozen weights (Select-N-Fold) instead of dropping them (ablation) | Fed-PLoRA · CoLA (IID) | accuracy | Improved |
| adapt PLoRA to rsLoRA (Fed-PrsLoRA) (ablation) | Fed-PLoRA · CoLA (IID), homogeneous | accuracy | Improved |
| Fed-PLoRA vs FedIT rank R on GLUE average | FedIT (R) · GLUE (IID) average | score: 81.08 -> 79.89 | Hurt |
Closest papers by meaning
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.94)
- FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · similarity 0.93)
- Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.93)
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA (NeurIPS 2025 · similarity 0.93)
- Beyond Parameter Aggregation: Semantic Consensus for Federated Fine-Tuning of LLMs (NeurIPS 2026 · similarity 0.91)
- Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
