Research map / Federated learning and privacy / Personalized and heterogeneous FL
dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes dFLMoE, a decentralized federated learning framework where clients exchange lightweight head models directly and fuse them with a client-specific attention-based mixture of experts for medical data analysis.
Problem
Centralized federated learning in medical settings causes knowledge damage during server-side aggregation and depends on the stability of a central server, while existing decentralized FL work mostly addresses security rather than the algorithm.
Approach
Clients share only lightweight head models peer-to-peer (no central aggregation), and each client treats local and received heads as experts combined by a cross-attention mixture of experts with a feature space transform, enabling client-specific fusion that tolerates model heterogeneity.
Key results
- On BreaKHis multi-resolution classification with homogeneous ResNet5 models, dFLMoE reaches average ACC 0.8319 vs 0.7362 for LG-FedAvg and 0.8083 for MH-pFLID.
- On Sleep-EDF time-series classification under heterogeneous models, dFLMoE achieves average ACC 0.9136 and MF1 0.8986, above MH-pFLID (0.8801 ACC).
- On polyp segmentation with heterogeneous U-Net variants, dFLMoE reaches average Dice 0.7813 vs 0.7146 for FedSM under the heterogeneous setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use decentralized dFLMoE instead of MH-pFLID | BreaKHis classification, homogeneous models · BreaKHis classification, Average | ACC: 0.8083 -> 0.8319 | Helped |
| use decentralized dFLMoE instead of MH-pFLID | BreaKHis classification, heterogeneous models · BreaKHis classification, Average | ACC: 0.8583 -> 0.8880 | Helped |
| use dFLMoE instead of FedRep | BreaKHis super-resolution, homogeneous models · BreaKHis super-resolution, Average | PSNR: 25.49 -> 28.61 | Helped |
| use dFLMoE instead of Only Local Training | BreaKHis super-resolution, heterogeneous models · BreaKHis super-resolution, Average | PSNR: 25.43 -> 29.30 | Helped |
| use dFLMoE instead of MH-pFLID | Sleep-EDF homogeneous models · Sleep-EDF, Average | ACC: 0.8700 -> 0.9041 | Helped |
| use dFLMoE instead of MH-pFLID | Sleep-EDF heterogeneous models · Sleep-EDF, Average | ACC: 0.8801 -> 0.9136 | Helped |
| use dFLMoE instead of MH-pFLID | BreaKHis breast cancer classification · BreaKHis breast cancer classification, Average | ACC: 0.9016 -> 0.9163 | Helped |
| use dFLMoE instead of MH-pFLID | ODIR-5K ocular disease recognition · ODIR-5K, Average | ACC: 0.7310 -> 0.7647 | Helped |
| use dFLMoE instead of LC-Fed | Polyp segmentation with homogeneous models · Polyp segmentation, Average | Dice: 0.6772 -> 0.7813 | Helped |
| use dFLMoE instead of MH-pFLID | Polyp segmentation with heterogeneous models · Polyp segmentation, Average | Dice: 0.7671 -> 0.7806 | Helped |
| add the designed MoE module (ablation) | dFLMoE · Sleep-EDF time-series classification | ACC: 0.8731 -> 0.9041 | Helped |
| add feature space transform (ablation) | dFLMoE · Sleep-EDF time-series classification | ACC: 0.8812 -> 0.9041 | Helped |
Surprising findings
- Using the entire local model as experts gave only limited gains over sharing heads (e.g., segmentation Dice 0.7921 vs 0.7806), while greatly increasing shared parameters.
Limitations
- Experiments are limited to five medical tasks and the authors note that the work reports only 'state-of-the-art' comparisons on these tasks.
- Sharing heads still requires N(N-1) peer-to-peer transmissions, which is more communication than centralized FL's 2N transmissions.
- Robustness evaluation under disconnects covers only one medical classification and one segmentation task.
Experimental setup
Datasets: BreaKHis Sleep-EDF ODIR-5K Polyp segmentation dataset (four centers)
Benchmarks: BreaKHis Sleep-EDF ODIR-5K Polyp segmentation dataset
Metrics: ACC MF1 PSNR SSIM Dice
Baselines: FedAvg SCAFFOLD FedProx Ditto APFL FedRep LG-FedAvg MH-pFLID FedMD FedDF pFedDF DS-pFL
Builds on: FedAvg MH-pFLID Ditto FedRep
Closest papers by meaning
- FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis (AAAI 2025 · similarity 0.89)
- MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis (ICML 2024 · similarity 0.88)
- Decentralized Directed Collaboration for Personalized Federated Learning (CVPR 2024 · similarity 0.86)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.86)
- SemiDFL: A Semi-Supervised Paradigm for Decentralized Federated Learning (AAAI 2025 · similarity 0.86)
- Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.