Research map / Federated learning and privacy / Personalized and heterogeneous FL
Tensor-aggregated LoRA in Federated Fine-tuning
Zhixuan Li, Binqian Xu, Xiangbo Shu, Jiachao Zhang, Yazhou Yao, Guo-Sen Xie, Jinhui Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace zero-padding alignment with PAA-strategy (optimal transport) (ablation) | Te-LoRA · MMLU Alpaca-GPT4, heterogeneous | accuracy: 44.77 -> 46.16 | Improved |
| replace FedAvg weighted average with T2M-strategy (ablation) | Te-LoRA · MMLU Alpaca-GPT4, heterogeneous | accuracy: 45.53 -> 46.16 | Improved |
| use T2M2 instead of T2M1 (ablation) | Te-LoRA · MMLU Alpaca-GPT4, heterogeneous | accuracy: 45.30 -> 46.16 | Improved |
| use T2M2 instead of combined T2M3 (ablation) | Te-LoRA · MMLU Alpaca-GPT4, heterogeneous | accuracy: 45.99 -> 46.16 | No clear effect |
| alternating-freeze training with PAA-strategy but without T2M, versus FlexLoRA (ablation) | Te-LoRA · MMLU Alpaca-GPT4, heterogeneous | accuracy: 45.48 -> 45.53 | Improved |
| use federated fine-tuning (Te-LoRA) instead of local-only training | Llama2-7B LoRA · MMLU Alpaca-GPT4, homogeneous | accuracy: 38.66 -> 45.86 | Improved |
| move from homogeneous to heterogeneous ranks for Zero-padding | Zero-padding · MMLU Wizard | accuracy: 22.03 -> 21.86 | Hurt |
| move from homogeneous to heterogeneous ranks for Te-LoRA | Te-LoRA · MMLU Alpaca-GPT4 | accuracy: 45.86 -> 46.16 (Alpaca-GPT4), 28.44 -> 28.37 (Dolly) | Mixed |
| alternate-freeze A and B, uploading only one module per round | Te-LoRA · communication cost per round | communication cost | Improved |
| lower LoRA rank to 4 for all clients (ablation) | Local LoRA · MMLU / MT-Bench | performance | Hurt |
Closest papers by meaning
- FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · similarity 0.92)
- Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.91)
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA (NeurIPS 2025 · similarity 0.91)
- Selective Aggregation for Low-Rank Adaptation in Federated Learning (ICLR 2025 · similarity 0.90)
- Improving LoRA in Privacy-preserving Federated Learning (ICLR 2024 · similarity 0.90)
- FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
