Research map / Federated learning and privacy / Personalized and heterogeneous FL

MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis

ICML 2024 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Personalization with global-local trade-off · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MH-pFLID, a public-data-free federated learning framework for model-heterogeneous clients that exchanges a lightweight messenger model via attention-based receiver/transmitter modules for knowledge injection and distillation.

Problem

Federated learning with heterogeneous client architectures and non-IID medical data, where existing knowledge-distillation methods require public datasets that are costly and raise privacy concerns.

Approach

A lightweight messenger model with a codebook-style body and head carries shared information; client-specific attention-based receiver modules inject messenger knowledge into local models, and transmitter modules distill local knowledge back into the messenger, removing the need for any public dataset.

Key results

• On breast cancer classification with heterogeneous models, MH-pFLID reaches 0.9006 ACC and 0.8465 MF1 average versus 0.8450 ACC for the best baseline KT-pFL (0.7739 MF1).
• On polyp segmentation, MH-pFLID achieves 0.7604 average Dice versus 0.7146 for FedSM and 0.6897 for local training.
• On Sleep-EDF time-series classification, MH-pFLID averages 0.8801 ACC versus 0.8621 for KT-pFL.

Measured effects reported in the paper

Surprising findings

• Generalizability tests show that the FCN client's accuracy on one other client's data drops (-10.2% on Client1) after messenger training, indicating not uniform gains.

Limitations

• Validated only on classification and segmentation tasks; detection, registration, and 3D reconstruction are untested.
• Some results are mixed: a generalizability test shows FCN accuracy drops on one client.
• Reported hyperparameters and hardware details are limited; compute scale is not stated.

Experimental setup

Datasets: BreaKHis OCT2017 Sleep-EDF Polyp segmentation dataset

Benchmarks: BreaKHis OCT2017 Sleep-EDF Polyp segmentation dataset

Metrics: ACC MF1 Dice

Baselines: FedAvg FedAvg+FT SCAFFOLD SCAFFOLD+FT FedProx FedProx+FT Ditto APFL FedRep LG-FedAvg FedMD FedDF

Builds on: FedMD FedDF KT-pFL FedAvg SCAFFOLD

Closest papers by meaning

• dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis (CVPR 2025 · similarity 0.88)
• FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis (AAAI 2025 · similarity 0.88)
• pFedClub: Controllable Heterogeneous Model Aggregation for Personalized Federated Learning (NeurIPS 2024 · similarity 0.87)
• FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models (CVPR 2025 · similarity 0.87)
• pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning (AAAI 2025 · similarity 0.86)
• Federated Learning via Input-Output Collaborative Distillation (AAAI 2024 · similarity 0.86)