Research map / Federated learning and privacy / Personalized and heterogeneous FL
Investigating Social Bias Propagation in Federated Fine-tuning of Large Language Models
Jiaxu Zhao, Meng Fang, Mingze Zhong, Shunfeng Zheng, Ling Chen, Mykola Pechenizkiy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BAMA bias-aware aggregation | FedAvg, LLaMA 3 · 1B/2C | CrowS-Pairs bias (dAvg): 21.06 -> 20.01 | Improved |
| add BAMA bias-aware aggregation | FedAvg, LLaMA 3 · 1B/2C | StereoSet bias (dAvg): 19.14 -> 17.96 | Improved |
| add BAMA bias-aware aggregation | FedAdam, Phi 3 · 1B/2C | CrowS-Pairs bias (dAvg): 18.03 -> 16.63 | Improved |
| add BAMA bias-aware aggregation | FedAdam, Phi 3 · 1B/2C | StereoSet bias (dAvg): 12.63 -> 10.42 | Improved |
| add BAMA bias-aware aggregation | FedProx, Gemma 2 2B · 1B/2C | CrowS-Pairs bias (dAvg): 17.44 -> 16.10 | Improved |
| add BAMA bias-aware aggregation | FedAvg, Gemma 2 2B · 1B/2C | CrowS-Pairs bias (dAvg): 17.52 -> 15.14 | Improved |
| use smaller model (Gemma 2 2B vs LLaMA 3 8B) with BAMA | BAMA · 1B/2C | CrowS-Pairs bias (dAvg): 2.38 points vs 1.05 points | Improved |
| use FedAdam instead of FedAvg | federated fine-tuning · Phi 3, 3C | CrowS-Pairs bias (dAvg): 16.67 -> 13.12 | Improved |
| use FedProx instead of FedAdam | federated fine-tuning · LLaMA 3, Mix 1B/2C | CrowS-Pairs bias (dAvg): 20.60 -> 21.36 | Hurt |
| use FedAdam instead of full fine-tuning | LLaMA 3 · 3B biased data | CrowS-Pairs bias (dAvg): 21.22 -> 19.54 | Improved |
| use LoRA instead of full fine-tuning | LLaMA 3 · 3B biased data | CrowS-Pairs bias (dAvg): 21.22 -> 20.84 | Improved |
| mix biased and clean data within each client instead of segregating | federated fine-tuning · all models and FL algorithms | bias scores: 5%-7% higher bias | Hurt |
Closest papers by meaning
- FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models (NeurIPS 2025 · similarity 0.91)
- FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations (NeurIPS 2024 · similarity 0.88)
- FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models (NeurIPS 2024 · similarity 0.88)
- FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models (ICML 2024 · similarity 0.88)
- Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models (ICML 2025 · similarity 0.87)
- Differentially Private Federated Low Rank Adaptation Beyond Fixed-Matrix (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
