Research map / Multi-agent systems and games / LLM agents
Weaving in the Clouds: Achieving Synergistic Collaboration among LLM Agents via Federated Learning
Jiaxing Zhao, Hongbin Xie, Yuzhen Lei, Xuan Song, Zhuoran Shi, Lianxin Li, Shuangxue liu, Haoran Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes federated learning for collaboration among LLM agents in multi-agent workflows so that agents can learn jointly without pooling distributed private data.
Problem
Multi-agent LLM systems need data distributed across organizations that cannot be centrally pooled due to privacy, IP, and compliance constraints; existing federated learning methods do not fit agent collaboration on workflow-structured tasks.
Approach
Applies federated learning to LLM agent collaboration so that expert agents synergize while data stays local; the abstract is truncated, so the specific mechanism is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VCLayer to FedAvg-LoRA (ablation) | FedAvg-LoRA · MSCoRe Automotive (Llama3-8B) | METEOR: 10.73 -> 17.93 | Improved |
| add MoE router to FedAvg-LoRA (ablation) | FedAvg-LoRA · MSCoRe Automotive (Llama3-8B) | METEOR: 10.73 -> 17.52 | Improved |
| add DPO alignment to FedAvg-LoRA (ablation) | FedAvg-LoRA · MSCoRe Automotive (Llama3-8B) | METEOR: 10.73 -> 21.04 | Improved |
| add DPO to VCLayer + Router (ablation) | FedWave (w/o DPO) · MSCoRe Automotive (Llama3-8B) | METEOR: 18.40 -> 24.83 | Improved |
| add VCLayer and Router on top of DPO (ablation) | DPO-only model · MSCoRe Automotive (Llama3-8B) | METEOR: 21.04 -> 24.83 | Improved |
| add consistency loss Lcons (ablation) | FedWave · MSCoRe Automotive (Llama3-8B) | METEOR: 24.28 -> 24.83 | Improved |
| add continuity loss Lcont (ablation) | FedWave · MSCoRe Automotive (Llama3-8B) | METEOR: 24.30 -> 24.83 | Improved |
| add positional loss Lpos (ablation) | FedWave · MSCoRe Automotive (Llama3-8B) | METEOR: 24.41 -> 24.83 | Improved |
| add load-balancing and entropy regularizers on the router (ablation) | FedWave · MSCoRe Automotive (Llama3-8B) | METEOR: 24.17 -> 24.83 | Improved |
| use FedWave instead of FedAvg | FedWave · MSCoRe Automotive (Llama3-8B) | METEOR: 10.73 -> 24.83 | Improved |
| use FedWave instead of FedAvg | FedWave · FinQA | F1: 5.58 -> 3.33 | Hurt |
| use FedWave instead of FedAvg | FedWave · MuSiQue | EM: 3.10 -> 7.20 | Improved |
Closest papers by meaning
- Helmsman: Autonomous Synthesis of Federated Learning Systems via Collaborative LLM Agents (ICLR 2026 · similarity 0.87)
- Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning (NeurIPS 2025 · similarity 0.86)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.86)
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA (NeurIPS 2025 · similarity 0.85)
- Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning (NeurIPS 2024 · similarity 0.85)
- TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
