Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedHera: Towards Drift-Resilient Federated Fine-tuning with Heterogeneous Resources
Ke Xiao, Qiyuan Wang, Christos Anagnostopoulos, Zhuoran Tan, Wenhao Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decouple downloaded rank r_tot from trainable rank r_train with prefix gating (frozen tail) instead of coupled ranks | FedHera vs FlexLoRA/FedHL · Alpaca (Skewed) | evaluation loss: 1.186 (FedHL) -> 1.140 | Improved |
| decouple rank reception from training | FedHera · E2E NLG (Skewed), Qwen3-0.6B | absolute drift: 2.991 (FlexLoRA) -> 2.060 | Improved |
| decoupling versus best coupled baseline FedHL | FedHera · E2E NLG (Skewed), Qwen3-0.6B | absolute drift: 2.072 -> 2.060 | No clear effect |
| increase local epochs from 1 to 5 (ablation) | FedHera · E2E NLG, Qwen3-0.6B | relative drift | Mixed |
| increase local epochs E from 1 to 7 for coupled baselines vs FedHera (ablation) | FlexLoRA, FedHL, FedHeLLo · E2E NLG | absolute drift | Hurt |
| use energy-aware water-filling rank allocation instead of uniform or random allocation (ablation) | FedHera · matched budgets | fidelity | Improved |
| add Adaptive Tail Warm-up (ATW) for the frozen tail (ablation) | FedHera · early rounds | training stability | Improved |
| use standard aggregation instead of residual aggregation (ablation) | FedHera · reasoning/generation tasks | accuracy | No clear effect |
| adapt all linear modules rather than a subset (ablation) | FedHera · ablation | performance | Improved |
| use FedHeLLo (layer pruning heterogeneity) instead of rank-adaptive methods | baseline · Alpaca (Skewed) | evaluation loss: 1.186 (FedHL) vs 1.617 | Hurt |
Closest papers by meaning
- Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning (NeurIPS 2026 · similarity 0.91)
- Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes (ICML 2024 · similarity 0.89)
- FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · similarity 0.87)
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA (NeurIPS 2025 · similarity 0.87)
- Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models (ICML 2025 · similarity 0.87)
- Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrix (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
