Research map / Federated learning and privacy / Communication-efficient FL
A Primal-Dual Algorithm for Hybrid Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyFDCA, a primal-dual (Fenchel dual, CoCoA-style) algorithm for hybrid federated learning where clients hold subsets of both samples and features, with homomorphic-encryption-based aggregation, and proves convergence to the centralized solution.
Problem
Hybrid federated learning, where clients hold subsets of both samples and features, lacks convergent algorithms that reach the centralized solution, and naive extensions of FedAvg and existing methods (HyFEM) perform poorly.
Approach
Extends CoCoA-style dual coordinate ascent to doubly partitioned data (samples and features across clients), with encrypted dual variables and secure inner products so the server cannot reconstruct client data, and proves convergence to the centralized optimum under complete and incomplete client participation.
Key results
- HyFDCA achieves lower relative loss and higher validation accuracy than FedAvg and HyFEM in 69 of 72 comparisons across settings.
- HyFDCA beats baselines in 36 of 36 comparisons with respect to outer iterations and 33 of 36 with respect to wall time.
- Convergence proofs show O(1/t) rate under complete participation and in horizontal FL with random client subsets, and O(log t / t) for vertical FL with H/N=1, matching or exceeding FedAvg and FedBCD rates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HyFDCA instead of FedAvg or HyFEM | Hybrid FL on MNIST, News20, and Covtype · Hybrid FL experiments | relative loss: 69 of 72 comparisons | Helped |
| Use HyFDCA instead of FedAvg or HyFEM | Hybrid FL on MNIST, News20, and Covtype · Hybrid FL experiments | validation accuracy: 69 of 72 comparisons | Helped |
| Use HyFDCA instead of FedAvg or HyFEM | Hybrid FL on MNIST, News20, and Covtype · Equal wall time experiments | relative loss: 33 of 36 comparisons | Helped |
| Use HyFDCA instead of FedAvg or HyFEM | Hybrid FL on MNIST, News20, and Covtype · Equal wall time experiments | validation accuracy: 33 of 36 comparisons | Helped |
| Use HyFDCA instead of FedAvg or HyFEM | Hybrid FL on MNIST, News20, and Covtype · Equal outer iterations experiments | relative loss: 36 of 36 comparisons | Helped |
| Use HyFDCA instead of FedAvg or HyFEM | Hybrid FL on MNIST, News20, and Covtype · Equal outer iterations experiments | validation accuracy: 36 of 36 comparisons | Helped |
| Omit the second SecureInnerProduct step (ablation) | HyFDCA practical experimental variant · Hybrid FL experiments | empirical performance | No clear effect |
Surprising findings
- Naively extended FedAvg performs poorly in the hybrid setting, showing horizontal/vertical algorithms do not directly transfer.
- Homomorphic encryption cost is the most expensive component of HyFDCA's per-iteration time.
Limitations
- Homomorphic encryption was not actually performed; encryption time was estimated from published benchmarks.
- Convergence proofs for hybrid setting with incomplete participation are not given; only horizontal (random subsets) and vertical (cyclic) special cases are covered.
- Experiments use a practical variant of the local method with hinge loss only, and per-iteration cost is higher than FedAvg and HyFEM.
Experimental setup
Datasets: MNIST News20 binary Covtype binary
Benchmarks: MNIST News20 binary Covtype binary
Metrics: relative loss function value validation accuracy wall time outer iterations
Baselines: FedAvg HyFEM
Builds on: CoCoA D3CA FedAvg FedBCD HyFEM
Closest papers by meaning
- A-FedPD: Aligning Dual-Drift is All Federated Primal-Dual Learning Needs (NeurIPS 2024 · similarity 0.88)
- Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable (NeurIPS 2025 · similarity 0.87)
- FedLMT: Tackling System Heterogeneity of Federated Learning via Low-Rank Model Training with Theoretical Guarantees (ICML 2024 · similarity 0.87)
- One Arrow, Two Hawks: Sharpness-aware Minimization for Federated Learning via Global Model Trajectory (ICML 2025 · similarity 0.87)
- Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability (NeurIPS 2024 · similarity 0.87)
- Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.