Research map / Federated learning and privacy / Personalized and heterogeneous FL

Federated Modality-Specific Encoders and Multimodal Anchors for Personalized Brain Tumor Segmentation

AAAI 2024 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated medical and multimodal learning · 18 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedMEMA, a federated learning framework for brain tumor segmentation with missing-modality clients that uses modality-specific shared encoders, a server fusion decoder, personalized client decoders, and multi-anchor multimodal prototypes calibrated via cross-attention.

Problem

Federated learning for multimodal medical imaging where participants hold only subsets of MRI modalities (inter-modal heterogeneity), requiring both a full-modal global server model and personalized missing-modal client models.

Approach

Exclusive federated encoder per modality with a server-side multimodal fusion decoder; the server extracts multiple K-means anchors per class from fused features and distributes them to clients, which calibrate their missing-modal features via scaled dot-product cross-attention (LACCA).

Key results

• On BraTS 2020 test (setting 1), FedMEMA reaches 62.01% average client mDSC and 82.11% server mDSC, versus 60.25% and 80.23% for the next best method FedIoT.
• In setting 2, FedMEMA achieves 63.59% client average and 83.27% server mDSC on the test set.
• Ablation on the validation set shows multi-anchor representations (3 anchors per class) outperform mono-anchor (64.13% vs 61.91% client average).

Measured effects reported in the paper

Surprising findings

• Specializing only normalization parameters (FedNorm) yields slight client gains but decreases server performance.
• The RFNet baseline, despite being robust to missing modalities, performs comparably to local training because it does not use extra client data.

Limitations

• Focuses on extreme monomodal clients; hetero-modal clients only addressed in supplementary experiments.
• Ignores intramodal heterogeneity across institutions.
• Assumes equal data sizes across clients with no dynamic weight assignment.

Experimental setup

Datasets: BraTS 2020 BraTS 2018

Benchmarks: BraTS 2020 BraTS 2018

Metrics: mean Dice similarity coefficient (mDSC)

Baselines: Local models RFNet FedAvg PerFL FedNorm CreamFL FedMSplit FedIoT

Builds on: FedAvg RFNet FedIoT CreamFL FedNorm

Closest papers by meaning

• Dual-calibrated Co-training Framework for Personalized Federated Semi-Supervised Medical Image Segmentation (AAAI 2025 · similarity 0.86)
• Semantic-guided Masked Mutual Learning for Multi-modal Brain Tumor Segmentation with Arbitrary Missing Modalities (AAAI 2025 · similarity 0.85)
• FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models (CVPR 2025 · similarity 0.85)
• Incomplete Multi-modal Brain Tumor Segmentation via Learnable Sorting State Space Model (CVPR 2025 · similarity 0.85)
• TMFormer: Token Merging Transformer for Brain Tumor Segmentation with Missing Modalities (AAAI 2024 · similarity 0.85)
• Overcoming Heterogeneous Data in Federated Medical Vision-Language Pre-training: A Triple-Embedding Model Selector Approach (AAAI 2025 · similarity 0.84)