Research map / Federated learning and privacy / Personalized and heterogeneous FL

FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection

NeurIPS 2024 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated medical and multimodal learning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedMEKI, a federated benchmark and platform that injects multi-modal, multi-task medical knowledge from private client data into a server-side medical foundation model (MMedLM-2) via a LLaVA-style adapter, evaluated with 16 baselines.

Problem

Scaling medical foundation models with diverse modalities (images, ECG signals, lab/vital tabular data) without centralizing privacy-sensitive hospital data, where client models have different architectures from the server foundation model and existing medical FMs handle only task-specific inputs.

Approach

Clients train modality-specific encoders and task decoders on private data and upload their encoder weights; the server averages them (FedAvg/FedProx) and uses the aggregated encoder with a linear projection plus task prompt embeddings to inject this knowledge into a frozen LLM, with only the server encoder updated on public data.

Key results

• Federated-scaled FedAvg^F_m reaches 95.48% accuracy on lung opacity detection in multi-task setting, while base MMedLM-2 cannot handle image or signal inputs at all.
• Scaled models underperform traditional FL baselines on training tasks; e.g., single-task ECG detection F1 is 18.74 for FedAvg^F_s vs 74.50 for FedAvg_s.
• Zero-shot validation generation results are weak: MedVQA BLEU about 1.20 and signal noise clarification BLEU 0.04-0.06 for the scaled models.

Measured effects reported in the paper

Surprising findings

• Federated-scaled foundation models perform worse than traditional federated baselines on the training tasks they were trained on.
• Zero-shot generation performance on validation tasks remains very low (e.g., BLEU below 0.1 for signal noise clarification).

Limitations

• Performance trade-off: federated-scaled foundation models lag traditional FL models on training tasks.
• Zero-shot performance on validation tasks is unsatisfactory, especially for generation tasks.
• No error bars or statistical significance reported; results from single runs.

Experimental setup

Datasets: RSNA Pneumonia Detection Challenge 2018 COVQU PTB-XL MIMIC-III CheXpert VQA-RAD ECG-QA

Benchmarks: FedMEKI training tasks (lung opacity, COVID-19, ECG abnormal, mortality) FedMEKI validation tasks (enlarged cardiomediastinum, sepsis, MedVQA, signal noise clarification)

Metrics: accuracy precision recall F1 BLEU ROUGE METEOR

Baselines: FedAvg FedProx MMedLM-2 FedAvg+ (global fine-tuning) FedProx+ (global fine-tuning)

Builds on: FedAvg FedProx LLaVA FLamby

Closest papers by meaning

• FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis (AAAI 2025 · similarity 0.86)
• FedPIA – Permuting and Integrating Adapters Leveraging Wasserstein Barycenters for Finetuning Foundation Models in Multi-Modal Federated Learning (AAAI 2025 · similarity 0.86)
• MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis (ICML 2024 · similarity 0.85)
• dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis (CVPR 2025 · similarity 0.85)
• Incongruent Multimodal Federated Learning for Medical Vision and Language-based Multi-label Disease Detection (AAAI 2025 · similarity 0.83)
• FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models (CVPR 2025 · similarity 0.83)