Research map / Federated learning and privacy / Personalized and heterogeneous FL
Language-Guided Transformer for Federated Multi-Label Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedLGT, a federated Transformer for multi-label image classification that combines CLIP-derived universal label embeddings with client-aware masked label embeddings, evaluated on FLAIR, MS-COCO and PASCAL VOC.
Problem
Federated learning for multi-label image classification, where label co-occurrence differs across clients (label distribution skew), so naive aggregation of local models degrades the global model.
Approach
Adapts the centralized C-Tran label-mask Transformer to FL by replacing learnable label embeddings with frozen CLIP text embeddings (universal label embeddings) so local models stay aligned, and by masking only classes the global model is uncertain about (client-aware masked label embedding) for local updates.
Key results
- On FLAIR coarse-grained, FedLGT reaches 60.90 C-AP and 55.10 C-F1 vs. 56.00 and 43.10 for FedC-Tran and 40.63 C-AP for FedAvg.
- On FLAIR fine-grained, FedLGT achieves 10.60 C-AP vs. 3.30 for FedC-Tran.
- Universal label embedding gives most of the gain (FedC-Tran + ULE 59.70 C-AP vs. 56.00 baseline), while CA-MLE alone adds only 0.1 C-AP on coarse FLAIR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FedLGT instead of FedC-Tran | FLAIR coarse-grained · FLAIR coarse-grained | C-AP: 56.00 -> 60.90 | Helped |
| Use FedLGT instead of FedC-Tran | FLAIR fine-grained · FLAIR fine-grained | C-AP: 3.30 -> 10.60 | Helped |
| Use FedLGT instead of FedC-Tran | MS-COCO · MS-COCO | C-AP: 76.70 -> 78.30 | Helped |
| Use FedLGT instead of FedC-Tran | PASCAL VOC · PASCAL VOC | C-AP: 89.60 -> 90.80 | Helped |
| Add CA-MLE to FedC-Tran without ULE (ablation) | FedC-Tran · FLAIR coarse-grained | C-AP: 56.00 -> 56.10 | No clear effect |
| Add ULE to FedC-Tran without CA-MLE (ablation) | FedC-Tran · FLAIR coarse-grained | C-AP: 56.00 -> 59.70 | Helped |
| Add CA-MLE alongside ULE (ablation) | FedLGT · FLAIR coarse-grained | C-AP: 59.70 -> 60.90 | Helped |
| Add CA-MLE to FedC-Tran without ULE (ablation) | FedC-Tran · FLAIR fine-grained | C-AP: 3.30 -> 5.50 | Helped |
| Add ULE to FedC-Tran without CA-MLE (ablation) | FedC-Tran · FLAIR fine-grained | C-AP: 3.30 -> 7.20 | Helped |
| Add CA-MLE alongside ULE (ablation) | FedLGT · FLAIR fine-grained | C-AP: 7.20 -> 10.60 | Helped |
| Set uncertainty margin ε to 0.02 instead of 0.05 (ablation) | CA-MLE · FLAIR coarse-grained | C-AP: 59.10 -> 60.90 | Helped |
| Use CLIP rather than GloVe for ULE (ablation) | FedLGT · FLAIR coarse-grained | C-AP: 57.90 -> 60.90 | Helped |
Surprising findings
- CA-MLE alone yields negligible improvement (0.1 C-AP on coarse FLAIR), with the gain coming mostly from ULE.
- Standard FedC-Tran degrades sharply on fine-grained FLAIR (10x lower C-AP than centralized C-Tran would suggest), i.e., label relations corrupted by aggregation.
Limitations
- Absolute performance on fine-grained FLAIR remains low (C-F1 2.30), and the gap to centralized results is still large.
- Gains from CA-MLE are small and sensitive to the uncertainty margin; reported results rely on a single seed and a fixed sampling strategy.
- Evaluated only with ResNet-18 backbone and 50 or 500 communication rounds.
Experimental setup
Datasets: FLAIR MS-COCO PASCAL VOC
Benchmarks: FLAIR MS-COCO PASCAL VOC
Metrics: C-AP O-AP C-P C-R C-F1 O-P O-R O-F1
Baselines: FedAvg FedC-Tran ResNet C-Tran
Builds on: C-Tran FedAvg CLIP FedProx FedBN
Closest papers by meaning
- Federated Partial Label Learning with Local-Adaptive Augmentation and Regularization (AAAI 2024 · similarity 0.87)
- (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning (NeurIPS 2024 · similarity 0.87)
- FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning (AAAI 2024 · similarity 0.86)
- FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels (AAAI 2024 · similarity 0.86)
- Exploiting Label Skews in Federated Learning with Model Concatenation (AAAI 2024 · similarity 0.86)
- Little Is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.