Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedTGP, a prototype-based heterogeneous federated learning method that learns trainable global prototypes on the server with an adaptive-margin contrastive loss, improving prototype separability over FedProto's weighted averaging.
Problem
Heterogeneous federated learning where clients have different model architectures and non-IID data; naive weighted averaging of client prototypes on the server yields global prototypes with shrunken inter-class margins, hurting client performance and leaking class distribution information.
Approach
Replace weighted-averaged global prototypes with trainable global prototypes optimized on the server via contrastive learning against client prototypes, with an adaptive margin set per iteration to the maximum inter-class prototype margin (capped by a threshold tau), so no client-model or class-count information is needed.
Key results
- FedTGP outperforms all six baselines on four datasets by up to 9.08% accuracy, and improves FedProto by up to 13.85% in the standard setting.
- In the most heterogeneous HtFE8-HtC4 scenario on Cifar100, FedTGP surpasses FedProto by 18.96% accuracy.
- Ablation on Cifar100 (practical setting): FedTGP reaches 46.94% vs. 36.34% for FedProto, with adaptive margin improving over fixed margin (43.46%) and standard contrastive loss (40.11%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FedTGP instead of FedKD | HtFE8 federated learning · Cifar10, pathological | accuracy: 87.32$\pm$0.31 -> 90.02$\pm$0.30 | Helped |
| use FedTGP instead of FedGen | HtFE8 federated learning · Cifar100, pathological | accuracy: 58.26$\pm$0.36 -> 61.86$\pm$0.30 | Helped |
| use FedTGP instead of FedGen | HtFE8 federated learning · Flowers102, pathological | accuracy: 59.90$\pm$0.15 -> 68.98$\pm$0.43 | Helped |
| use FedTGP instead of FedKD | HtFE8 federated learning · Tiny-ImageNet, pathological | accuracy: 32.64$\pm$0.36 -> 34.56$\pm$0.27 | Helped |
| use FedTGP instead of FedKD | HtFE8 federated learning · Cifar10, practical | accuracy: 86.45$\pm$0.10 -> 88.15$\pm$0.43 | Helped |
| use FedTGP instead of FedDistill | HtFE8 federated learning · Cifar100, practical | accuracy: 41.54$\pm$0.08 -> 46.94$\pm$0.12 | Helped |
| replace adaptive-margin contrastive learning with standard contrastive loss (ablation) | FedTGP · Cifar100, practical | accuracy: 46.94 -> 40.11 | Hurt |
| replace adaptive margin with a fixed margin (ablation) | FedTGP · Cifar100, practical | accuracy: 46.94 -> 43.46 | Hurt |
| remove the server processing model F (ablation) | FedTGP · Cifar100, practical | accuracy: 46.94 -> 40.37 | Hurt |
| set margin threshold τ to 1000 instead of 100 (ablation) | FedTGP · Cifar100, practical | accuracy: 46.94 -> 46.09 | Hurt |
| increase server training epochs S from 100 to 1000 (ablation) | FedTGP · Cifar100, practical | accuracy: 46.94 -> 47.01 | No clear effect |
Surprising findings
- Naive weighted averaging of client prototypes shrinks the inter-class prototype margin below the best client margins, degrading global prototype quality.
- Increasing the number of client training epochs reduces accuracy for FML and FedKD, which use auxiliary models.
Limitations
- Adaptive margin threshold tau and server epochs S are hyperparameters; accuracy drops at tau=1000.
- Evaluated only on image classification with small image datasets and simulated clients.
Experimental setup
Datasets: Cifar10 Cifar100 Tiny-ImageNet Flowers102 Fashion-MNIST
Benchmarks: Cifar10 Cifar100 Tiny-ImageNet Flowers102 Fashion-MNIST
Metrics: test accuracy (%) communication cost (M parameters per iteration) FLOPs
Baselines: LG-FedAvg FedGen FML FedKD FedDistill FedProto
Builds on: FedProto contrastive learning ArcFace FedAvg
Closest papers by meaning
- Relaxed Contrastive Learning for Federated Learning (CVPR 2024 · similarity 0.88)
- Beyond Federated Prototype Learning: Learnable Semantic Anchors with Hyperspherical Contrast for Domain-Skewed Data (AAAI 2025 · similarity 0.87)
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.87)
- FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning (AAAI 2025 · similarity 0.86)
- An Upload-Efficient Scheme for Transferring Knowledge From a Server-Side Pre-trained Generator to Clients in Heterogeneous Federated Learning (CVPR 2024 · similarity 0.86)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.