Research map / Federated learning and privacy / Personalized and heterogeneous FL
Federated Learning over Connected Modes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FLoCo, a federated learning method that trains a low-loss solution simplex of last-layer weights and assigns each client a subregion of it based on projected gradient signals, improving both global and personalized accuracy.
Problem
Statistical heterogeneity in federated learning causes slow global training from conflicting gradients and poor local (personalized) performance, with a typical trade-off between global and local accuracy.
Approach
Leverages linear mode connectivity: a solution simplex of model endpoints is trained jointly across clients, where each client samples models from an assigned subregion of the simplex derived by projecting its gradient signals onto the standard simplex (PCA plus Riesz s-energy projection). Simplex learning is applied only to the last layer, and an optional fine-tuning variant (FLoCo+) adds Ditto-s
Key results
- FLoCo reaches average local accuracy of 85.90% on pre-trained ResNet-18 CIFAR-10 5-Fold versus 76.94% for FedAvg local, and global accuracy of 62.93% on CifarCNN 5-Fold versus 60.36% for FedAvg.
- Time-to-accuracy improvements over FedAvg reach up to x5.5 (global, CifarCNN 5-Fold).
- FLoCo+ reports lower local ECE than Ditto on CifarCNN 5-Fold (17.69 vs 19.13), and worst-5% client accuracy improves up to 17% over FedAvg.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace global training with Floco solution-simplex training and client subregion assignment | CIFAR-10 5-Fold, CifarCNN · CIFAR-10 5-Fold, CifarCNN global | accuracy: 60.68 -> 62.93 | Helped |
| Use Floco client subregion personalization | CIFAR-10 5-Fold, CifarCNN · CIFAR-10 5-Fold, CifarCNN local | accuracy: 74.03 -> 71.78 | Hurt |
| Replace global training with Floco solution-simplex training and client subregion assignment | CIFAR-10 Dir(0.3), CifarCNN · CIFAR-10 Dir(0.3), CifarCNN global | accuracy: 60.74 -> 62.57 | Helped |
| Use Floco client subregion personalization | CIFAR-10 Dir(0.3), CifarCNN · CIFAR-10 Dir(0.3), CifarCNN local | accuracy: 76.42 -> 71.04 | Hurt |
| Apply simplex learning only to the last fully connected layer rather than all network parameters (ablation) | Floco · CIFAR-10 global and local | accuracy | No clear effect |
| Assign subregions earlier and reduce the subregion radius (ablation) | Floco · CIFAR-10 5-Fold, CifarCNN local | accuracy: 78.18% against 82.79% | Helped |
| Vary subregion assignment round and radius (ablation) | Floco · CIFAR-10 5-Fold, CifarCNN global | accuracy: 70.66% against 69.30% | No clear effect |
Surprising findings
- FedRoD and APFL fail to converge (around 4.95% accuracy) in several settings.
- Simplex training before subregion assignment already matches FedAvg global performance, with personalization gains coming from the subregion assignment.
Limitations
- Evaluated only in cross-silo settings with up to 100 clients; cross-device settings are not studied.
- Assumes stateful clients with enough data to group similar clients.
- No theoretical analysis of the approach is provided.
Experimental setup
Datasets: CIFAR-10 FEMNIST
Benchmarks: CIFAR-10 FEMNIST
Metrics: global test accuracy local test accuracy expected calibration error time-to-accuracy improvement worst 5% client accuracy
Baselines: FedAvg FedProx FedPer APFL Ditto FedRoD SuPerFed
Builds on: Ditto FedAvg simplex learning (Benton et al. 2021) SuPerFed
Closest papers by meaning
- FedECADO: A Dynamical System Model of Federated Learning (ICML 2025 · similarity 0.86)
- Low Precision Local Training is Enough for Federated Learning (NeurIPS 2024 · similarity 0.86)
- FedSMU: Communication-Efficient and Generalization-Enhanced Federated Learning through Symbolic Model Updates (ICML 2025 · similarity 0.85)
- Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization (ICML 2024 · similarity 0.85)
- Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.85)
- Rethinking the Flat Minima Searching in Federated Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.