Research map / Federated learning and privacy / Personalized and heterogeneous FL

FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Models

ICML 2025 · Federated learning and privacy › Personalized and heterogeneous FL · approach: Federated prompt and LoRA tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FOCoOp, a federated prompt learning framework that uses ID global, local and OOD prompts with bi-level distributionally robust optimization and semi-unbalanced optimal transport calibration to jointly improve generalization and OOD detection on CLIP.

Problem

Federated prompt learning for vision-language models trades off ID/covariate-shift generalization against OOD detection, and ID data heterogeneity across clients makes consistent OOD robustness hard.

Approach

Learn three prompt sets (ID global, ID local, OOD) per client with bi-level distributionally robust optimization (OT perturbation of global prompts, unbalanced OT perturbation of OOD prompts), and at the server align global and OOD prompts via semi-unbalanced optimal transport to calibrate global prompts and send distant OOD prompts back.

Key results

• On CIFAR-100 pathological non-overlap (K=10), FOCoOp reaches 93.85 ACC, 91.47 CACC, 19.50 FPR95 and 95.42 AUROC, versus 91.82 ACC and 44.26 FPR95 for the best baseline PromptFolio.
• On TinyImageNet (K=10), FOCoOp reaches 88.35 ACC and 96.56 AUROC, compared with 88.03 ACC and 87.34 AUROC for PromptFolio.
• Ablations show removing BOS or GOC degrades ACC and FPR95 on CIFAR-100, with removing GOC causing the larger drop.

Measured effects reported in the paper

Surprising findings

• Existing FPL methods such as PromptFolio achieve strong ID accuracy but fail badly at OOD detection (FPR95 above 40% on CIFAR-100), indicating the performance-robustness trade-off.

Limitations

• Experiments limited to CLIP ViT-B/16 and ResNet50 backbones.
• Heterogeneity is simulated via Dirichlet and pathological partitions rather than real federated data.
• Requires hyperparameter tuning of several coefficients (rho, alpha, tau1, tau2) with sensitivity shown per dataset.

Experimental setup

Datasets: CIFAR-100 TinyImageNet CIFAR-100-C TinyImageNet-C iNaturalist iSUN Places Texture Food101 DTD Caltech101 Flowers102

Benchmarks: CIFAR-100 TinyImageNet CIFAR-100-C TinyImageNet-C DomainNet Office-Caltech10 Food101 DTD Caltech101 Flowers102 OxfordPets

Metrics: ACC CACC FPR95 AUROC

Baselines: PromptFL FedOTP FedPGP PromptFolio FedLoCoOp FedGalLoP FedLAPT

Builds on: CoOp FedOTP MCM FedGalLoP

Closest papers by meaning

• Global and Local Prompts Cooperation via Optimal Transport for Federated Learning (CVPR 2024 · similarity 0.89)
• Harmonizing Generalization and Personalization in Federated Prompt Learning (ICML 2024 · similarity 0.89)
• FedPHA: Federated Prompt Learning for Heterogeneous Client Adaptation (ICML 2025 · similarity 0.88)
• Mixture of Experts Made Personalized: Federated Prompt Learning for Vision-Language Models (ICLR 2025 · similarity 0.88)
• FLiP: Towards Comprehensive and Reliable Evaluation of Federated Prompt Learning (NeurIPS 2025 · similarity 0.88)
• Federated Learning from Vision-Language Foundation Models: Theoretical Analysis and Method (NeurIPS 2024 · similarity 0.88)