Research map / Federated learning and privacy / Personalized and heterogeneous FL
Balancing Similarity and Complementarity for Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FedSaC, a federated learning framework that builds a client cooperation network by optimizing a weighted balance of model similarity and feature-subspace complementarity, evaluated on CIFAR-10/100 and CUB200-2011.
Problem
Statistical heterogeneity in federated learning makes choosing collaborators hard; existing personalized FL methods select collaborators by model similarity, which may not yield optimal cooperation when clients have complementary data.
Approach
Argues optimal cooperation requires balancing model similarity (conditional p(y|x) consistency) with feature-space complementarity (covariate shift in p(x)), measured via principal angles between SVD subspaces of local features, and optimizes a constrained client cooperation weight matrix on the server.
Key results
- On CIFAR-10 under Dirichlet high heterogeneity (alpha=0.1), FedSaC reaches 93.14% average accuracy vs. 84.28% for pFedGraph.
- On CIFAR-100 Homogeneous partition, FedSaC achieves 34.84% vs. 31.16% for pFedGraph and 31.10% for FedAvg.
- On multimodal CUB200-2011 (Dirichlet high), FedSaC gets 35.36% visual and 33.44% textual accuracy vs. 31.78% and 31.34% for pFedGraph.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| balance model similarity with feature complementarity instead of relying on similarity alone | FedSaC · CIFAR-10 and CIFAR-100, all reported heterogeneity partitions | accuracy: improving accuracy by 3% to 8% | Helped |
| balance model similarity with feature complementarity instead of similarity-based cooperation | FedSaC · CUB200-2011, visual and textual modalities across all partitions | accuracy | Helped |
| vary complementarity weight α from 0.6 to 1.2 (ablation) | FedSaC · CIFAR-10 partitions with complementarity | accuracy | Helped |
| vary complementarity weight α from 0.6 to 1.2 (ablation) | FedSaC · CIFAR-10 partitions with high heterogeneity | accuracy | Mixed |
| vary similarity weight β from 0.2 to 1.6 (ablation) | FedSaC · CIFAR-10 partitions | accuracy | Helped |
| vary local regularization weight λ (ablation) | FedSaC · CIFAR-100, four partitions | accuracy | Mixed |
| set representative subspace dimension k to 3 rather than k=1, 2, or 4 (ablation) | FedSaC · CIFAR-10, four partitions | accuracy | Helped |
Surprising findings
- Complementarity-only cooperation leads clients to largely abandon their initial models, which is disadvantageous for training.
- Similarity-only cooperation leaves clients mostly keeping their own models, reducing information gain.
Limitations
- Large-scale client settings require random subsetting of 10 clients per round to maintain convergence and efficiency.
- Hyperparameters alpha and beta depend on whether the dataset has complementarity; setting switches are manually chosen.
- Cross-modality cooperation is restricted to the classification layer, and complementarity is excluded there.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CUB200-2011
Benchmarks: CIFAR-10 CIFAR-100 CUB200-2011
Metrics: average accuracy
Baselines: Local FedAvg FedProx CFL pFedMe Ditto FedAMP FedRep pFedHN FedRoD kNN-Per pFedGraph
Builds on: pFedGraph FedAvg FedAMP Ditto
Closest papers by meaning
- FedCompetitors: Harmonious Collaboration in Federated Learning with Competing Participants (AAAI 2024 · similarity 0.89)
- FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.87)
- Bridging Model Heterogeneity in Federated Learning via Uncertainty-based Asymmetrical Reciprocity Learning (ICML 2024 · similarity 0.87)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.87)
- FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors (CVPR 2025 · similarity 0.87)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.