Research map / Federated learning and privacy / Personalized and heterogeneous FL
Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Drift
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedCCFA, a federated learning framework that clusters local classifiers per class and aligns client feature spaces with clustered feature anchors weighted by label-distribution entropy, to handle distributed concept drift under data heterogeneity.
Problem
Federated learning under data heterogeneity combined with distributed real concept drift, where clients have different conditional distributions P(Y|X) that change over time, which single-model FL methods handle poorly.
Approach
Decouple the network into extractor and classifier; cluster clients per class using their balanced classifier weights (MADD-style cosine distance with DBSCAN) and share aggregated class classifiers within clusters; build clustered feature anchors per cluster for contrastive feature alignment, with the alignment weight scaled by the entropy of the local label distribution.
Key results
- Under sudden drift with Dir(0.5) and 20 clients at full participation, FedCCFA reaches 89.50% on Fashion-MNIST, 76.95% on CIFAR-10 and 51.12% on CINIC-10, outperforming the second-best method by 2.48%, 9.50% and 0.06% on those datasets respectively (CINIC-10 margin is negligible; the paper reports F
- Under sudden drift with Dir(0.5) and 100 clients at 20% participation, FedCCFA reaches 89.39%, 73.21% and 52.62%, improving over the second-best by 1.14%, 8.48% and 3.34%.
- Ablation on CIFAR-10 under sudden drift (partial participation): balanced classifiers for clustering give 73.21% vs 68.39% with local classifiers, close to an oracle clustering at 73.55%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use balanced classifiers rather than local classifiers as clustering input (ablation) | FedCCFA · CIFAR-10, sudden drift, Dir(0.5), 100 clients | generalized accuracy: 68.39$\pm$0.71 -> 73.21$\pm$0.82 | Helped |
| use oracle clustering results (ablation) | FedCCFA · CIFAR-10, sudden drift, Dir(0.5), 100 clients | generalized accuracy: 73.21$\pm$0.82 -> 73.55$\pm$0.27 | Helped |
| add feature alignment (ablation) | FedCCFA · CIFAR-10, sudden drift, Dir(0.5), 100 clients | generalized accuracy: 71.40$\pm$0.43 -> 73.21$\pm$0.82 | Helped |
| use clustered rather than non-clustered feature anchors (ablation) | FedCCFA · CIFAR-10, sudden drift, Dir(0.5), 100 clients | generalized accuracy: 72.77$\pm$0.71 -> 73.21$\pm$0.82 | Helped |
| replace adaptive alignment weight with fixed weight 0.1 (ablation) | FedCCFA · CIFAR-10, Dir(0.01), 100 clients | accuracy: 62.46$\pm$0.92 -> 57.86$\pm$1.91 | Hurt |
| replace adaptive alignment weight with fixed weight 1.0 (ablation) | FedCCFA · CIFAR-10, Dir(0.01), 100 clients | accuracy: 62.46$\pm$0.92 -> 40.89$\pm$3.97 | Hurt |
| replace adaptive alignment weight with fixed weight 0.1 (ablation) | FedCCFA · CIFAR-10, Dir(0.5), 100 clients | accuracy: 74.44$\pm$0.26 -> 73.83$\pm$0.53 | Hurt |
| replace adaptive alignment weight with fixed weight 1.0 (ablation) | FedCCFA · CIFAR-10, Dir(0.5), 100 clients | accuracy: 74.44$\pm$0.26 -> 74.80$\pm$0.41 | Helped |
| change adaptive alignment scaling factor gamma (ablation) | FedCCFA · CIFAR-10, Dir(0.01), 100 clients | accuracy: 60.30$\pm$0.79; 61.29$\pm$0.59; 62.15$\pm$0.49 | Mixed |
Surprising findings
- Under no drift, FedCCFA and other decoupled methods (FedRep, FedBABU, FedPAC) lag single-model methods such as FedAvg, which the authors attribute to classifier-first training causing gradient attenuation into the extractor.
- Under severe heterogeneity Dir(0.01), training without feature alignment outperforms training with fixed alignment weight, showing alignment can impede main-task learning.
Limitations
- Balanced classifier training adds computation cost, which the authors acknowledge and hope to remove.
- Feature alignment can impede main-task learning under severe data heterogeneity, and the method is not best on CINIC-10 with full participation under Dir(0.1) sudden drift due to wrong clustering.
- Experiments are limited to label distribution skew and small CNNs on three image datasets.
Experimental setup
Datasets: Fashion-MNIST CIFAR-10 CINIC-10
Benchmarks: Fashion-MNIST CIFAR-10 CINIC-10
Metrics: generalized accuracy (average across clients at last round) accuracy decrease under drift
Baselines: FedAvg FedProx SCAFFOLD FedFM AdapFedAvg Flash pFedMe Ditto FedRep FedBABU FedPAC IFCA
Builds on: FedPAC FedFM FedDrift MADD
Closest papers by meaning
- Federated Learning with Sample-level Client Drift Mitigation (AAAI 2025 · similarity 0.88)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.86)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.86)
- A New Theoretical Perspective on Data Heterogeneity in Federated Optimization (ICML 2024 · similarity 0.86)
- FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts (NeurIPS 2025 · similarity 0.85)
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.