Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedLoGe, a federated long-tailed learning framework that jointly trains a shared backbone with a sparse fixed ETF classifier (SSE-C) and norm-based global/local classifier realignment (GLA-FR) to improve both generic and personalized models.
Problem
Federated long-tailed learning (Fed-LT) typically optimizes only a global generic model under globally long-tailed, locally heterogeneous data, neglecting personalized local model performance, while standard pFL assumes balanced global data.
Approach
Replace the dense ETF classifier with a static sparse ETF (SSE-C) that prunes noisy low-mean features, then realign a global auxiliary classifier by normalizing its class-vector norms and transfer its norms to personalized classifiers, leveraging neural collapse and classifier-norm/class-cardinality correlations.
Key results
- On CIFAR-100-LT (IF=100, alpha=0.5), FedLoGe reaches 0.4233 global and 0.7285 personalized accuracy, versus 0.3919 / 0.6919 for FedROD.
- On ImageNet-LT, FedLoGe achieves 0.356 global and 0.726 personalized accuracy, outperforming the best baseline Ratio Loss (0.351 GM) and FedROD (0.703 PM).
- Ablation on CIFAR-100-LT shows SSE-C plus GA-FR and LA-FR together gives 0.4350 GM and 0.7343 PM, compared with 0.3825 GM and 0.6421 PM for the dense ETF baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace dense ETF with SSE-C (ablation) | FedLoGe representation learning · CIFAR-100-LT (α=0.5, IF=100) | GM accuracy: 0.3825 -> 0.4175 | Helped |
| replace dense ETF with SSE-C (ablation) | FedLoGe representation learning · CIFAR-100-LT (α=0.5, IF=100) | PM accuracy: 0.6421 -> 0.6865 | Helped |
| add GA-FR (ablation) | SSE-C backbone · CIFAR-100-LT (α=0.5, IF=100) | GM accuracy: 0.4175 -> 0.4350 | Helped |
| add LA-FR (ablation) | SSE-C backbone · CIFAR-100-LT (α=0.5, IF=100) | PM accuracy: 0.6865 -> 0.7339 | Helped |
| add LA-FR after SSE-C and GA-FR (ablation) | FedLoGe · CIFAR-100-LT (α=0.5, IF=100) | PM accuracy: 0.6868 -> 0.7343 | Helped |
| add GA-FR after SSE-C and LA-FR (ablation) | FedLoGe · CIFAR-100-LT (α=0.5, IF=100) | GM accuracy: 0.4175 -> 0.4350 | Helped |
| add a projection layer (ablation) | FedLoGe · ImageNet-LT | All accuracy: 0.4249 -> 0.4004 | Hurt |
| add sparsification to ETF + projection + dot-regression loss (ablation) | ETF representation learning · ImageNet-LT | All accuracy: 0.4109 -> 0.4237 | Helped |
| use personalized classifier norm rather than real local distribution for LA-FR (ablation) | FedLoGe local alignment · CIFAR-100-LT (α=0.5, IF=100) | PM (Avg): 0.6200 -> 0.7273 | Helped |
| sparsify the fixed classifier (ablation) | fixed classifier initialization · ImageNet-LT | All accuracy: 0.3784 -> 0.4237 | Helped |
Surprising findings
- Pruning the smallest-mean (negligible) features barely affects performance, while pruning dominant features sharply degrades it.
- Using classifier norms to realign local models outperforms realignment with the real local class distribution (PM avg 0.7273 vs 0.6200).
Limitations
- Sparsity ratio and norm gamma are hyperparameters requiring sensitivity tuning (optimal gamma ~1.0 for CIFAR, 1.6 for ImageNet/iNaturalist).
- Experiments limited to image classification on ResNet backbones with simulated Dirichlet non-IID partitions.
- Privacy analysis is limited; norm analysis at client or server is not deeply examined.
Experimental setup
Datasets: CIFAR-10-LT CIFAR-100-LT ImageNet-LT iNaturalist-User-160k
Benchmarks: CIFAR-10-LT CIFAR-100-LT ImageNet-LT iNaturalist-User-160k
Metrics: global model accuracy personalized model accuracy many/medium/few-shot accuracy
Baselines: FedAvg FedProx FedBN FedPer FedRep Ditto FedROD FedBABU FedETF Ratio Loss
Builds on: FedETF FedRep FedBABU FedRoD MMA
Closest papers by meaning
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.89)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.89)
- Subgraph Federated Learning for Local Generalization (ICLR 2025 · similarity 0.88)
- Personalized Federated Learning via Feature Distribution Adaptation (NeurIPS 2024 · similarity 0.88)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.88)
- FedUV: Uniformity and Variance for Heterogeneous Federated Learning (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.