Research map / Federated learning and privacy / Personalized and heterogeneous FL
Learning Together Securely: Prototype-Based Federated Multi-Modal Hashing for Safe and Efficient Multi-Modal Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PFMH, a prototype-based federated multi-modal hashing framework that exchanges class prototypes instead of raw data, with prototype completion for missing classes, evaluated on Wikipedia, MIR-Flickr and NUS-WIDE.
Problem
Multi-modal hashing retrieval is usually centralized, raising privacy concerns; federated settings face parameter leakage, class imbalance and statistical heterogeneity across clients, and weak multi-modal fusion.
Approach
Clients learn class-wise prototypes from fused image-text features and send only these prototypes to the server; the server aggregates them into global prototypes and fills in missing classes for each client, and clients optimize hash codes with global and local prototype contrastive losses plus a hash optimization loss.
Key results
- On Wikipedia at 128 bits, PFMH reaches MAP 0.8126 versus 0.8112 for PT-FUCH and 0.8105 for FedProto.
- On NUS-WIDE at 128 bits, PFMH reaches MAP 0.8091 versus 0.7058 for PT-FUCH and 0.7009 for FedAvg.
- On MIR-Flickr at 16 bits, PFMH reaches MAP 0.8640 versus 0.8486 for PT-FUCH; PFMH is slightly below FedAvg at 128 bits (0.8789 vs 0.8819 for PFMH).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add local/global prototype contrast (ablation) | PFMH · Wikipedia, 64 bits | MAP: 0.7783 -> 0.7998 | Helped |
| add local/global prototype contrast (ablation) | PFMH · MIR-Flickr, 64 bits | MAP: 0.8734 -> 0.8778 | Helped |
| add local/global prototype contrast (ablation) | PFMH · NUS-WIDE, 64 bits | MAP: 0.7821 -> 0.7903 | Helped |
| add prototype enhancement (ablation) | PFMH · Wikipedia, 64 bits | MAP: 0.7984 -> 0.7998 | Helped |
| add prototype enhancement (ablation) | PFMH · MIR-Flickr, 64 bits | MAP: 0.8754 -> 0.8778 | Helped |
| add prototype enhancement (ablation) | PFMH · NUS-WIDE, 64 bits | MAP: 0.7887 -> 0.7903 | Helped |
| include similarity matching (ablation) | PFMH · Wikipedia, 64 bits | MAP: 0.7749 -> 0.7998 | Helped |
| include similarity matching (ablation) | PFMH · MIR-Flickr, 64 bits | MAP: 0.8616 -> 0.8778 | Helped |
| include similarity matching (ablation) | PFMH · NUS-WIDE, 64 bits | MAP: 0.7794 -> 0.7903 | Helped |
| include information preservation (ablation) | PFMH · Wikipedia, 64 bits | MAP: 0.7800 -> 0.7998 | Helped |
| include information preservation (ablation) | PFMH · MIR-Flickr, 64 bits | MAP: 0.8726 -> 0.8778 | Helped |
| include information preservation (ablation) | PFMH · NUS-WIDE, 64 bits | MAP: 0.7897 -> 0.7903 | Helped |
Surprising findings
- The federated PFMH outperforms its own centralized variant (CentralizedOur) on the reported benchmarks, which the authors attribute to client data diversity.
- On MIR-Flickr at 128 bits, the proposed method is slightly below FedAvg in the reported table (0.8819 for PFMH is higher than FedAvg's 0.8789, so the gap is in the other direction at some bit lengths; results are mixed across settings).
Limitations
- Evaluated only on three image-text datasets with simulated Dirichlet non-IID partitions over 10 clients.
- Privacy claims are qualitative; no formal privacy analysis or attack evaluation is provided.
- Hyperparameters alpha and beta show notable sensitivity according to the paper.
Experimental setup
Datasets: Wikipedia MIR-Flickr NUS-WIDE
Benchmarks: Wikipedia MIR-Flickr NUS-WIDE
Metrics: MAP Top-K precision
Baselines: FedAvg FedProto PT-FUCH BSTH GCIMH Local
Builds on: FedAvg FedProto PT-FUCH BSTH GCIMH
Closest papers by meaning
- Distribution-Consistency-Guided Multi-modal Hashing (AAAI 2025 · similarity 0.86)
- Dual Self-Paced Cross-Modal Hashing (AAAI 2024 · similarity 0.85)
- Asymmetric Cross-Modal Hashing Based on Formal Concept Analysis (AAAI 2025 · similarity 0.85)
- Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels (AAAI 2025 · similarity 0.85)
- Semi-Supervised Online Cross-Modal Hashing (AAAI 2025 · similarity 0.84)
- Statistical Model-driven Similarity Hashing: Bridging Modalities for Efficient Unsupervised Retrieval (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.