Research map / Federated learning and privacy / Personalized and heterogeneous FL
Adaptive Hyper-graph Aggregation for Modality-Agnostic Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HAMFL, a modality-agnostic federated learning framework that uses a modular local model, a global consensus prototype enhancer, and server-side adaptive hypergraph aggregation to handle modality and statistical heterogeneity across clients.
Problem
Federated learning where clients hold different data modalities (modality incompatibility) and non-IID data (statistical heterogeneity), with the server unaware of client modalities.
Approach
Server-side hypergraph whose hyperedges are initialized by k-means clustering on modality-specific parameters and on public-dataset accuracy profiles (distributional domain), with aggregation weights learned by a hypergraph diffusion network supervised by public-data accuracy; clients use a modular model with modality-specific, modality-shared, and personalized modules plus global prototype regula
Key results
- On MELD (A-V-T emotion recognition) the method reaches 54.8% average vs 53.1% for FDARN, about 1.7 points higher, with 1.0% gain cited for A-V-T in text.
- On EPIC-Kitchens with alpha=0.2, accuracy is 35.2% vs 28.3% for FedAvg (about 24.4% relative improvement).
- On UCF-101 video-flow, the method does not beat FDARN (flow accuracy 35.2 vs 34.1 for FDARN is higher than FDARN in table; authors note flow is weak), yet average accuracy is 41.6% on EPIC-Kitchens vs 40.7% for FDARN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HAMFL instead of FDARN | EPIC-Kitchens · EPIC-Kitchens | average accuracy: 40.7(±0.64) -> 41.6(±0.53) | Helped |
| Use HAMFL instead of CreamFL | UCF-101 · UCF-101 | average accuracy: 72.9(±1.62) -> 73.3(±1.24) | Helped |
| Use HAMFL instead of FDARN | MELD · MELD | average accuracy: 53.1(±1.39) -> 54.1(±1.43) | Helped |
| Use HAMFL instead of FDARN | EPIC-Kitchens, α=0.2 · EPIC-Kitchens, α=0.2 | average accuracy: 34.1(±0.61) -> 35.2(±0.65) | Helped |
| Use HAMFL instead of FDARN | EPIC-Kitchens, α=0.5 · EPIC-Kitchens, α=0.5 | average accuracy: 37.6(±0.38) -> 39.4(±0.46) | Helped |
| Use HAMFL instead of FDARN | EPIC-Kitchens, α=1 · EPIC-Kitchens, α=1 | average accuracy: 40.7(±0.64) -> 41.6(±0.53) | Helped |
| Use HAMFL instead of FDARN for the flow modality | UCF-101 · UCF-101 flow modality | accuracy | Hurt |
| Increase the public dataset size (ablation) | HAMFL on UCF-101 · UCF-101 | accuracy | Helped |
| Increase the public dataset size (ablation) | HAMFL on EPIC-Kitchens and MELD · EPIC-Kitchens and MELD | accuracy | Hurt |
| Vary the Modality Speculative Domain hyperedge count; best performance occurs at the number of modalities (ablation) | HAMFL on UCF-101 and MELD · UCF-101 and MELD | accuracy | Mixed |
| Vary the Distributional Speculative Domain hyperedge count; use µ2=5 on UCF-101 and µ2=3 on MELD (ablation) | HAMFL · UCF-101 and MELD | accuracy | Mixed |
Surprising findings
- The modality-agnostic method, without server knowledge of client modalities, performs close to or above modality-specific methods.
- Larger public datasets hurt performance on EPIC-Kitchens and MELD, attributed to long-tail class imbalance.
Limitations
- Acknowledged limitations in data availability and need for optimization, and challenges in evaluating FL generalization.
- Optical flow modality underperforms FDARN on UCF-101 due to sparse flow data and dimensionality reduction.
- Larger public datasets can bias toward majority classes on long-tailed datasets.
Experimental setup
Datasets: EPIC-Kitchens UCF-101 MELD
Benchmarks: EPIC-Kitchens UCF-101 MELD
Metrics: top-1 accuracy
Baselines: FedAvg FedProto FedLAW pFedGraph FedMSplit FDARN CreamFL SingleSet
Builds on: pFedGraph ED-HNN CreamFL FDARN FedProto
Closest papers by meaning
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.87)
- FedMBridge: Bridgeable Multimodal Federated Learning (ICML 2024 · similarity 0.86)
- Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data (NeurIPS 2025 · similarity 0.86)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.86)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.86)
- Modeling Inter-Intra Heterogeneity for Graph Federated Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.