Research map / Federated learning and privacy / Personalized and heterogeneous FL
Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fed-QSSL, a federated self-supervised learning framework where clients train low-bitwidth (heterogeneous bitwidth) SimCLR-style encoders and the server de-quantizes, loss-weighted aggregates, and re-quantizes models, with a convergence and representation-robustness analysis.
Problem
Federated learning where both local data distributions (non-IID) and client device bitwidth capabilities are heterogeneous, so local models have different precisions and expressiveness and cannot be aggregated directly.
Approach
Combine low-bit quantized training with distributed self-supervised learning on clients, and on the server de-quantize each client's low-bit model with fine-tuning on a small unlabeled global buffer, aggregate with loss-based softmax weights, then re-quantize to each client's bitwidth.
Key results
- On CIFAR-10 with 20% 4-bit/30% 6-bit/30% 8-bit/20% 12-bit clients, Fed-QSSL reaches 59.31 global and 82.50 local accuracy vs. 39.09/77.30 for Fed-SimCLR and 17.70/64.98 for FedAvg.
- On CIFAR-10 with 50% 6-bit and 50% 12-bit clients, Fed-QSSL achieves 62.26 global and 83.03 local accuracy, above Fed-SimCLR (39.12/77.43).
- On CIFAR-100 (50% 6-bit, 50% 12-bit), Fed-QSSL reaches 21.44 global and 30.91 local accuracy vs. 14.79/23.46 for Fed-SimCLR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Fed-QSSL instead of Fed-SimCLR | ResNet-18 federated learning · CIFAR-10, 20% 4-bit / 30% 6-bit / 30% 8-bit / 20% 12-bit | Global accuracy: 39.09 -> 59.31 | Helped |
| Use Fed-QSSL instead of Fed-SimCLR | ResNet-18 federated learning · CIFAR-10, 20% 4-bit / 30% 6-bit / 30% 8-bit / 20% 12-bit | Local accuracy: 77.30 -> 82.50 | Helped |
| Use Fed-QSSL instead of Fed-SimCLR | ResNet-18 federated learning · CIFAR-10, 50% 6-bit / 50% 12-bit | Global accuracy: 39.12 -> 62.26 | Helped |
| Use Fed-QSSL instead of Fed-SimCLR | ResNet-18 federated learning · CIFAR-10, 50% 6-bit / 50% 12-bit | Local accuracy: 77.43 -> 83.03 | Helped |
| Use Fed-QSSL instead of Fed-SimCLR | ResNet-18 federated learning · CIFAR-100, 50% 6-bit / 50% 12-bit | Global accuracy: 14.79 -> 21.44 | Helped |
| Use Fed-QSSL instead of Fed-SimCLR | ResNet-18 federated learning · CIFAR-100, 50% 6-bit / 50% 12-bit | Local accuracy: 23.46 -> 30.91 | Helped |
| Increase Dirichlet distribution parameter β (ablation) | Fed-QSSL · CIFAR-10 | Global accuracy | Helped |
| Increase Dirichlet distribution parameter β (ablation) | Fed-QSSL · CIFAR-10 | Local accuracy | Hurt |
| Decrease the global-buffer-to-local-dataset size ratio (ablation) | Fed-QSSL · CIFAR-10 | Global accuracy | Hurt |
| Decrease the global-buffer-to-local-dataset size ratio (ablation) | Fed-QSSL · CIFAR-10 | Local accuracy | No clear effect |
| Increase the number of local training epochs (ablation) | Fed-QSSL · CIFAR-10 | accuracy | Helped |
Surprising findings
- Fed-QSSL under low bitwidth outperforms some full-precision baselines in local accuracy (e.g., Fed-SimCLR Full 80.56 on CIFAR-10 is below Fed-QSSL 82.50 at low bitwidth).
Limitations
- Simulation only with 10 clients; large-scale settings left to future work.
- Theoretical analysis uses simplified linear SSL objective and assumes unbiased quantizers and a simplified server (no de-quantizer).
Experimental setup
Datasets: CIFAR-10 CIFAR-100
Benchmarks: CIFAR-10 CIFAR-100
Metrics: global accuracy local accuracy
Baselines: FedAvg FedProx FedPAQ Fed-SimCLR Fed-SimSiam
Builds on: FedAvg SimCLR SSQL FedPAQ DoReFa-Net
Closest papers by meaning
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.90)
- Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices (CVPR 2024 · similarity 0.90)
- Federated Learning with Heterogeneous LLMs: Integrating Small Student Client Models with a Large Hungry Model (AAAI 2025 · similarity 0.89)
- A Bayesian Approach for Personalized Federated Learning in Heterogeneous Settings (NeurIPS 2024 · similarity 0.89)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.89)
- LBI-FL: Low-Bit Integerized Federated Learning with Temporally Dynamic Bit-Width Allocation (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.