Research map / Federated learning and privacy / Personalized and heterogeneous FL
FedSC: Provable Federated Self-supervised Learning with Spectral Contrastive Objective over Non-i.i.d. Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedSC, a provable federated SSL method that shares DP-protected correlation matrices alongside model weights to enable inter-client contrast under the spectral contrastive objective.
Problem
Federated self-supervised learning over non-i.i.d. data, where the global SSL objective is not a weighted sum of local SSL objectives, so FedAvg does not minimize the true global objective.
Approach
Reformulate the spectral contrastive objective so that its global form decomposes into intra-client contraction, intra-client contrast and inter-client contrast; clients share DP-noised local correlation matrices (aggregated by the server) and use them in local objectives to approximate inter-client contrast, with a convergence guarantee to the centralized SC solution.
Key results
- FedSC achieves 80.06% on CIFAR10 and 58.35% on CIFAR100 under full participation, beating the best baseline FedCA (78.22%, 52.35%) and approaching centralized SC (90.21%, 64.32%).
- Under partial participation on CIFAR100 (4/20 clients), FedSC reaches 56.64% versus FedAvg+SC at 42.95%.
- With DP at epsilon=6 (delta=1e-2) on CIFAR100, FedSC retains 57.10% under full participation, only a modest drop from 58.35%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Share correlation matrices in addition to model weights (FedSC vs FedAvg+SC) | SC objective federated training · CIFAR100 20/20 and 4/20 participation | linear evaluation accuracy (%): +2.1 at full participation; +13.7 at 4/20 participation | Helped |
| Share correlation matrices in addition to model weights (FedSC vs FedAvg+SC) | SC objective federated training · SVHN and CIFAR10 with partial and full participation | linear evaluation accuracy (%): SVHN +1.3 (5/5), +1.1 (2/5); CIFAR10 +2.2 (10/10), +1.8 (2/10) | Helped |
| DP noise (eps=6 vs no DP) on shared correlation matrices (ablation) | FedSC · CIFAR10 10/10 participation | linear evaluation accuracy (%): -0.64 vs no DP | No clear effect |
| DP noise on shared correlation matrices at eps=6 and eps=12 (ablation) | FedSC · CIFAR100 4/20 participation | linear evaluation accuracy (%): eps=6: -1.77 and eps=12: -0.88 vs no DP; both still well above FedAvg+SC (42.95) | Mixed |
| Apply DP noise to shared correlation matrices (eps=3 to 6) (ablation) | FedSC · CIFAR10 and SVHN | linear evaluation accuracy (%): small declines from no-DP (about 0.6 to 1.9 points) but still above FedAvg+SC | No clear effect |
| Use SC objective with FedSC framework instead of BYOL under FedAvg | FedAvg federated SSL · CIFAR10 10/10 participation | linear evaluation accuracy (%): +9.68 over FedAvg+BYOL | Helped |
| Partial client participation (2/10 vs 10/10 on CIFAR10) for FedSC | FedSC · CIFAR10 | linear evaluation accuracy (%): -2.94 (80.06 to 77.12) | No clear effect |
| Partial participation on CIFAR100 (20/20 to 4/20) for FedAvg+SC | FedAvg+SC · CIFAR100 | linear evaluation accuracy (%): -13.29 (56.24 to 42.95) | Hurt |
Surprising findings
- FedEMA is best on SVHN full participation (91.87%) and FedAvg+SC overfits or plateaus at high error floors as rounds grow, while FedSC consistently improves KNN accuracy.
Limitations
- Global objective is not exactly a weighted sum of local objectives, so convergence rate relies on bias terms that do not vanish with finite batch size or views
- DP noise and aging of shared correlation matrices introduce error; privacy guarantee is only for data-level DP on the correlation matrices, not encoders
- Experiments limited to small image benchmarks (SVHN, CIFAR10, CIFAR100) with ResNet backbones
Experimental setup
Datasets: SVHN CIFAR10 CIFAR100
Benchmarks: SVHN CIFAR10 CIFAR100
Metrics: linear evaluation accuracy KNN accuracy
Baselines: FedAvg+BYOL FedAvg+SC FedU FedEMA FedX FedCA Centralized SC
Builds on: spectral contrastive loss FedAvg BYOL FedCA
Closest papers by meaning
- Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity (AAAI 2024 · similarity 0.88)
- Subgraph Federated Learning via Spectral Methods (NeurIPS 2025 · similarity 0.87)
- Little Is Enough: Boosting Privacy by Sharing Only Hard Labels in Federated Semi-Supervised Learning (AAAI 2025 · similarity 0.87)
- Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off (ICML 2025 · similarity 0.87)
- (FL)$^2$: Overcoming Few Labels in Federated Semi-Supervised Learning (NeurIPS 2024 · similarity 0.87)
- A Mutual Information Perspective on Federated Contrastive Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.