Research map / Federated learning and privacy / Communication-efficient FL

Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data

NeurIPS 2024 · Federated learning and privacy › Communication-efficient FL · approach: Federated optimization algorithms · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a federated algorithm for nonconvex optimization over compact smooth submanifolds with heterogeneous client data, using Riemannian stochastic gradients, projection onto the manifold, local updates and client-drift correction terms, with sublinear convergence to a neighborhood of a first-order optimal point.

Problem

Federated nonconvex optimization over compact smooth submanifolds (e.g., Stiefel manifold) where averaging local models leaves the manifold and existing manifold FL methods rely on costly exponential/inverse-exponential maps and parallel transport, with heterogeneous client data.

Approach

Replace exponential-map-based averaging and parallel transport with a projection operator onto the manifold, compute correction terms locally (no extra communication), and use proximal smoothness of the manifold to prove Lipschitz-like properties of the projection and convergence with multiple local updates and full participation.

Key results

• Theorem gives (1/R) sum E||G||^2 <= 8Omega/(sqrt(n) eta tau R) + 64 sigma^2/(n tau b), i.e., O(1/(sqrt(n) tau R)) rate for tau>1 with full participation.
• On kPCA with MNIST, the proposed method reaches high accuracy with much lower communication quantity and running time than RFedavg, RFedprox and RFedSVRG; RFedavg and RFedprox suffer from client drift.
• On LRMC, the method matches RFedSVRG per communication round and is faster in communication quantity and running time.

Measured effects reported in the paper

Limitations

• Projection onto some manifolds has no closed form, requiring approximate projections.
• Step size depends on the proximal smoothness constant, which is hard to compute for general manifolds.
• Only full client participation is analyzed; partial participation remains open.

Experimental setup

Datasets: MNIST

Benchmarks: MNIST kPCA low-rank matrix completion synthetic setting

Metrics: gradient norm ||grad f|| communication quantity running time

Baselines: RFedavg RFedprox RFedSVRG

Builds on: FedAvg FedProx SCAFFOLD-style correction RFedSVRG

Closest papers by meaning

• Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization (ICLR 2025 · similarity 0.88)
• Scaffold with Stochastic Gradients: New Analysis with Linear Speed-Up (ICML 2025 · similarity 0.87)
• Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable (NeurIPS 2025 · similarity 0.87)
• Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization (AAAI 2025 · similarity 0.87)
• Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees (ICLR 2025 · similarity 0.87)
• Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis (NeurIPS 2024 · similarity 0.87)