Research map / Federated learning and privacy / Personalized and heterogeneous FL
On the Importance and Applicability of Pre-Training for Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize FedAvg with ImageNet-pre-trained weights instead of random weights | FedAvg with ResNet20 · CIFAR-10 (α=0.3) | accuracy: 82.7 -> 90.8 | Helped |
| Initialize FedAvg with FPS-pre-trained weights instead of random weights | FedAvg with ResNet18 · Tiny-ImageNet (α=0.3) | accuracy: 42.4 -> 45.7 | Helped |
| Use FPS fractal-pair self-supervision rather than random initialization | FedAvg · iNaturalist-GEO-10K | accuracy: 20.9 -> 27.8 | Helped |
| Use FPS fractal-pair self-supervision rather than random initialization | FedAvg · Cityscapes | mIoU: 41.0 -> 64.9 | Helped |
| Use FPS fractal-pair training rather than standard fractal images with SimSiam (ablation) | Synthetic pre-training for FedAvg · CIFAR-10 (α=0.1) | accuracy: 77.4 -> 80.5 | Helped |
| Replace FPS self-supervision with fractal multi-label supervision (ablation) | Synthetic pre-training for FedAvg · CIFAR-10 (α=0.1) | accuracy: 80.5 -> 73.0 | Hurt |
| Add federated self-supervised pre-training before supervised FedAvg (ablation) | FedAvg with random initialization · CIFAR-10 (α=0.1) | accuracy: 74.4 -> 79.8 | Helped |
| Add federated self-supervised pre-training before supervised FedAvg (ablation) | FedAvg with FPS initialization · CIFAR-10 (α=0.1) | accuracy: 79.9 -> 82.2 | Helped |
| Add federated self-supervised pre-training before supervised FedAvg (ablation) | FedAvg with real pre-trained initialization · CIFAR-10 (α=0.1) | accuracy: 87.9 -> 88.0 | No clear effect |
| Pre-train with FPS rather than ImageNet weights | FedAvg for chest X-ray diagnosis · Chest X-ray diagnosis (α=0.3) | accuracy: 69.1 -> 74.8 | Helped |
| Use ImageNet pre-training rather than random initialization | FedAvg for Sent140 · Sent140 | accuracy: 75.4 -> 83.4 | Helped |
| Use pre-training rather than random initialization | FedAvg for Shakespeare next-character prediction · Shakespeare next-character prediction | accuracy: 46.4 -> 52.6 | Helped |
Closest papers by meaning
- Rethinking the Starting Point: Collaborative Pre-Training for Federated Downstream Tasks (AAAI 2025 · similarity 0.87)
- FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning (AAAI 2024 · similarity 0.86)
- FedImpro: Measuring and Improving Client Update in Federated Learning (ICLR 2024 · similarity 0.86)
- FedUV: Uniformity and Variance for Heterogeneous Federated Learning (CVPR 2024 · similarity 0.86)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.86)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.