Research map / Federated learning and privacy / Personalized and heterogeneous FL
UniFLoW: Universal Multi-Modal Federated LoRA Fine-Tuning Framework with Analytical Aggregation
Haoyuan Liang, Zhiyu Ye, Jielong Tang, Yang Yang, Shilei Cao, Guowen Li, Fei Hu, Zhiwei Zhang, Haohuan Fu, Juepeng Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace FedSA-LoRA aggregation with FedA2-LoRA | LoRA on RoBERTa · GLUE avg | accuracy: 90.39 -> 90.50 | Improved |
| replace rsLoRA aggregation with FedA2-rsLoRA | rsLoRA on RoBERTa · GLUE avg | accuracy: 89.16 -> 90.68 | Improved |
| replace VeRA aggregation with FedA2-VeRA | VeRA on RoBERTa · GLUE avg | accuracy: 86.86 -> 88.02 | Improved |
| replace LoRA aggregation with FedA2-LoRA | LoRA · PandaGPT image test, image+audio clients | Acc: 64.89 -> 75.00 | Improved |
| add II stage training (encoder first, then LLM) (ablation) | LoRA · PandaGPT image test, image-only clients | Acc: 64.68 -> 66.67 | Improved |
| add II stage training (ablation) | FFA-LoRA · PandaGPT image test, image+audio clients | Acc: 63.15 -> 70.58 | Improved |
| add II stage training (ablation) | FedA2-LoRA · PandaGPT image test, image-only clients | Acc: 69.02 -> 72.22 | Improved |
| add II stage training (ablation) | FedSA-LoRA · HeySQuAD audio test, audio-only clients | Acc: 53.84 -> 53.84 | No clear effect |
| add Tikhonov regularization (lambda=0.1) to B recovery (ablation) | FedA2-LoRA · PandaGPT image test, image+audio clients | Acc: 65.09 -> 76.19 | Improved |
| add Tikhonov regularization to B recovery (ablation) | FedA2-LoRA · PandaGPT image test, image+audio clients | FBERT: 80.96 -> 85.69 | Improved |
| use FedA2-LoRA with heterogeneous ranks (zero-padding) instead of LoRA | LoRA · PandaGPT image test, heterogeneous-rank clients | FBERT: 82.34 -> 84.46 | Improved |
| replace LoRA aggregation with FedA2-LoRA using LLaMA-1B and UniBind | LoRA · PandaGPT image test, LLaMA-1B + UniBind | Acc: 63.15 -> 65.00 | Improved |
Closest papers by meaning
- FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · similarity 0.91)
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.91)
- FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models (NeurIPS 2025 · similarity 0.90)
- Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.90)
- Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources (NeurIPS 2024 · similarity 0.90)
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
