Research map / Federated learning and privacy / Privacy attacks and secure learning

MFL-Owner: Ownership Protection for Multi-modal Federated Learning via Orthogonal Transform Watermark

AAAI 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Private federated aggregation · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MFL-Owner, a black-box watermarking framework for multi-modal federated learning CLIP models that injects per-client orthogonal linear transforms trained on OoD trigger sets to enable ownership verification and leaker tracing.

Problem

Protecting ownership and enabling traceability of global multi-modal federated learning models deployed as Embedding-as-a-Service, without degrading downstream retrieval and classification performance and without watermark conflicts across clients.

Approach

Decouples watermarking from FL training by inserting a near-orthogonal linear transform after the CLIP encoders, trained per client on trigger sets of Gaussian-noised OoD images paired with LLM-generated distinct texts; ownership and leaker are identified by cosine and Euclidean distance differences versus the clean model.

Key results

• MFL-Owner keeps downstream accuracy within 0.10% to 2.81% of the unwatermarked model on classification, versus 8.36% to 35.26% drops for EmbM and 9.05% to 35.38% for Fed-VLPM/o.
• MFL-Owner achieves an average watermark detection rate of 97.11%, compared with 96.57% for Fed-VLPM/o.
• Traceability heatmaps show each client's watermarked model brings only its own trigger-set pairs closer, enabling leaker identification across five clients.

Measured effects reported in the paper

Surprising findings

• Fed-VLPM/o yields a larger verification score than MFL-Owner, but at the cost of substantially greater downstream performance degradation.

Limitations

• Watermark becomes ineffective when the global model's embedding space changes through continued updates, since the orthogonal matrix depends on original embeddings.
• Detection performance decreases as the number of trigger image-text pairs grows.
• Experiments simulate identical data across clients and use a single CLIP model setting.

Experimental setup

Datasets: Visual Genome Flickr30k CIFAR-10 CIFAR-100 ImageNet-1k VOC2007

Benchmarks: Flickr30k CIFAR-10 CIFAR-100 ImageNet-1k VOC2007

Metrics: recall@5 accuracy p-value detection rate verification score

Baselines: Original (unwatermarked MFL CLIP) EmbM Fed-VLPM/o

Builds on: VLPMarker FedAvg EmbMarker

Closest papers by meaning

• FedGMark: Certifiably Robust Watermarking for Federated Graph Learning (NeurIPS 2024 · similarity 0.85)
• ZeroMark: Towards Dataset Ownership Verification without Disclosing Watermark (NeurIPS 2024 · similarity 0.84)
• pFedES: Generalized Proxy Feature Extractor Sharing for Model Heterogeneous Personalized Federated Learning (AAAI 2025 · similarity 0.83)
• Enhancing Storage and Computational Efficiency in Federated Multimodal Learning for Large-Scale Models (ICML 2024 · similarity 0.83)
• Federated Learning via Input-Output Collaborative Distillation (AAAI 2024 · similarity 0.83)
• EntropyMark: Towards More Harmless Backdoor Watermark via Entropy-based Constraint for Open-source Dataset Copyright Protection (CVPR 2025 · similarity 0.83)