Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedMBridge: Bridgeable Multimodal Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedMBridge, a topology-aware hypernetwork held on the server that generates heterogeneous multimodal client models from their computational graphs, enabling federated sharing across architecture-personalized clients.
Problem
Multimodal federated learning when clients have structurally distinct, non-splittable architectures and different multimodal fusion strategies, plus statistical (non-IID) heterogeneity.
Approach
Represents each client's architecture as a computational graph and uses a GNN-based layer-role encoder plus a hypernetwork weight generator on the server, so clients share knowledge through an implicit latent space rather than aggregating unaligned weights.
Key results
- On MnistAMF, FedMBridge reaches 95.78% ACC versus 92.67% for the best baseline (HyperPFL w/ Prune) and 91.83% for Local.
- On SceneAMF, FedMBridge achieves 83.92% ACC, versus 81.32% for FedGKD and 76.56% for Local.
- FedMBridge's per-round communication time (37.4 s on SceneAMF, 29.5 s on MnistAMF) is lower than the pruning-based HyperPFL (117.9 s and 140.2 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FedMBridge instead of FedGKD | AMFL methods · SceneAMF | ACC (%): 81.32 ± 0.83 -> 83.92 ± 0.95 | Helped |
| use FedMBridge instead of FedGKD | AMFL methods · ObjectAMF | ACC (%): 93.82 ± 0.98 -> 94.64 ± 0.94 | Helped |
| use FedMBridge instead of FedGKD | AMFL methods · EmotionAMF | ACC (%): 73.97 ± 0.86 -> 75.96 ± 0.83 | Helped |
| use FedMBridge instead of FedGKD | AMFL methods · MnistAMF | ACC (%): 93.91 ± 0.63 -> 95.78 ± 0.61 | Helped |
| remove the GNN-based layer-role encoder (L=0) (ablation) | FedMBridge · MnistAMF | ACC: 95.78 -> 87.75 | Hurt |
| replace concatenation with sum for role-task fusion (ablation) | FedMBridge · MnistAMF | ACC: 95.78 -> 95.51 | No clear effect |
| increase client selection from 0.25 to 0.40 (ablation) | FedMBridge · MnistAMF | ACC: 95.78 -> 96.15 | Helped |
| increase client selection from 0.25 to 0.40 (ablation) | FedMBridge · MnistAMF | COT: 29.5 -> 36.2 | Hurt |
| add feature-sharing knowledge distillation (β=0.01) (ablation) | FedMBridge · MnistAMF | ACC: 95.78 -> 95.92 | Helped |
| add feature-sharing knowledge distillation (β=0.05) (ablation) | FedMBridge · MnistAMF | ACC: 95.78 -> 96.53 | Helped |
| add feature-sharing knowledge distillation (β=0.01) (ablation) | FedMBridge · MnistAMF | COT: 29.5 -> 41.9 | Hurt |
| add feature-sharing knowledge distillation (β=0.05) (ablation) | FedMBridge · MnistAMF | COT: 29.5 -> 41.9 | Hurt |
Surprising findings
- Feature-sharing baselines such as FedDistill were sensitive to modality gap and statistical heterogeneity in AMFL.
- Removing the GNN layer-role encoder (L=0) drops MnistAMF accuracy from 95.78% to 87.75%, showing the role encoder is the most important component.
Limitations
- Feature-sharing integration gives only marginal gains, and balancing parameter and feature sharing is left open.
- Feature-sharing baselines require a public dataset from 5% of clients, which the authors note raises privacy risk.
Experimental setup
Datasets: NYU-v2 ModelNet40 CMU-MOSEI AVMnist MultiMnist
Benchmarks: SceneAMF ObjectAMF EmotionAMF MnistAMF
Metrics: ACC COT (communication time per round, s)
Baselines: Local FedDistill FedGKD HeteroFL w/ Mask HyperPFL w/ Prune APFL w/ Prune
Builds on: HyperPFL DARTS HeteroFL FedGKD tensor fusion
Closest papers by meaning
- Enhancing Storage and Computational Efficiency in Federated Multimodal Learning for Large-Scale Models (ICML 2024 · similarity 0.89)
- Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data (NeurIPS 2025 · similarity 0.87)
- Adaptive Hyper-graph Aggregation for Modality-Agnostic Federated Learning (CVPR 2024 · similarity 0.86)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.86)
- Large Language Models Enhanced Personalized Graph Neural Architecture Search in Federated Learning (AAAI 2025 · similarity 0.86)
- FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.