Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
Haoran Zhang, Dongjun Kim, Seohyeon Cha, Haris Vikalo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Procrustes rotational alignment before factor averaging | FedIT · GLUE avg (N=50, r=4) | accuracy: 0.7680 -> 0.8734 | Improved |
| add rotational alignment | FedIT · MNLI aggregation error | aggregation error: 3.98e-3 -> 1.48e-4 | Improved |
| replace alignment with random orthogonal rotations (ablation) | FedRot-LoRA · MNLI | accuracy: 0.876 -> 0.3181 | Hurt |
| alternate alignment between A and B instead of aligning only A (ablation) | FedRot-LoRA · SST-2, QNLI, MNLI | accuracy: MNLI 0.861 -> 0.876 | Improved |
| alternate alignment between A and B instead of aligning only B (ablation) | FedRot-LoRA · QNLI | accuracy: 0.806 -> 0.926 | Improved |
| use intermediate soft rotation lambda rather than lambda=1.0 or 0 (ablation) | FedRot-LoRA · MNLI, QQP | accuracy | Improved |
| increase LoRA rank to 16 and 24 (ablation) | RoLoRA · MNLI (N=3) | accuracy: 0.869 (r=8) -> 0.722 (r=16) | Hurt |
| increase LoRA rank from 4 to 24 (ablation) | FedRot-LoRA · MNLI (N=3) | accuracy: 0.876 -> 0.872 | No clear effect |
| increase data heterogeneity (h from 100 to 0.5) (ablation) | FedRot-LoRA · MNLI | accuracy: 0.884 -> 0.876 | Hurt |
| increase number of clients from 3 to 50 (ablation) | FedIT · GLUE avg | accuracy: 0.8712 -> 0.7680 | Hurt |
| increase number of clients from 3 to 50 (ablation) | FedRot-LoRA · GLUE avg | accuracy: 0.8932 -> 0.8734 | Hurt |
| add rotational alignment | FedIT · GLUE QQP (N=3) | accuracy: 0.771 -> 0.842 | Improved |
Closest papers by meaning
- Selective Aggregation for Low-Rank Adaptation in Federated Learning (ICLR 2025 · similarity 0.89)
- FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · similarity 0.88)
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA (NeurIPS 2025 · similarity 0.88)
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.87)
- Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models (ICML 2025 · similarity 0.87)
- RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
