Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Learning from Pre-Trained Models: A Contrastive Learning Approach
Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, Jing Jiang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fuse multiple fixed pre-trained backbones and train with FedPCL | Lightweight federated learning · Digit-5, feature shift non-IID, average | accuracy: 51.93(1.18) -> 55.34(0.34) | Improved |
| Use both global and local prototypes for local contrastive training (ablation) | FedPCL · Digit-5, feature shift non-IID | accuracy: 54.69(0.14) -> 55.34(0.34) | Improved |
| Use both global and local prototypes for local contrastive training (ablation) | FedPCL · Digit-5, label shift non-IID | accuracy: 44.75(1.77) -> 45.35(1.58) | Improved |
| Use both global and local prototypes for local contrastive training (ablation) | FedPCL · Digit-5, feature & label shift non-IID | accuracy: 43.79(4.09) -> 45.22(3.65) | Improved |
| Replace prototype-wise contrastive loss with cross-entropy (ablation) | FedPCL · Digit-5, feature & label shift non-IID | accuracy: 45.22(3.65) -> 32.98(3.44) | Hurt |
| Replace prototype-wise contrastive loss with cross-entropy plus ProtoDist (ablation) | FedPCL · Digit-5, feature & label shift non-IID | accuracy: 45.22(3.65) -> 43.94(3.02) | Hurt |
| Replace prototype-wise contrastive loss with supervised contrastive loss using local embeddings only (ablation) | FedPCL · Digit-5, feature & label shift non-IID | accuracy: 45.22(3.65) -> 42.18(3.25) | Hurt |
| Increase the number of fixed backbones from 1 to 3 (ablation) | FedPCL · Digit-5, feature & label shift non-IID | accuracy: 42.87(1.47) -> 45.22(3.65) | Improved |
| Increase the number of fixed backbones from 2 to 3 (ablation) | FedPCL · Digit-5, feature & label shift non-IID | accuracy: 44.49(1.75) -> 45.22(3.65) | Improved |
| Add noise to communicated prototypes (ablation) | FedPCL · Office-10, label shift non-IID | accuracy: 42.13(0.77) -> 39.93(3.48) (Laplace, p=0.1); 42.13(0.77) -> 40.04(2.09) (Gaussian, p=0.1) | Hurt |
| Use three fixed backbones instead of one (ablation) | Lightweight federated learning · Digit-5, feature shift non-IID, FedPCL | accuracy: 52.83(0.21) -> 55.34(0.34) | Improved |
Closest papers by meaning
- Relaxed Contrastive Learning for Federated Learning (CVPR 2024 · similarity 0.92)
- FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning (AAAI 2024 · similarity 0.91)
- A Mutual Information Perspective on Federated Contrastive Learning (ICLR 2024 · similarity 0.89)
- Efficient Federated Learning via Clients-to-Server Knowledge Distillation (Student Abstract) (AAAI 2025 · similarity 0.88)
- PeFLL: Personalized Federated Learning by Learning to Learn (ICLR 2024 · similarity 0.88)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
