Research map / Federated learning and privacy / Communication-efficient FL

Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems

ICML 2024 · Federated learning and privacy › Communication-efficient FL · approach: Federated optimization algorithms · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts the mu^2-SGD corrected-momentum method with Gaussian noise to federated SCO, giving DP guarantees with optimal rates and linear compute for untrusted and trusted parameter-server settings.

Problem

Differentially private federated stochastic convex optimization in centralized (parameter-server) silo settings, where prior optimal population-loss DP methods have super-linear computational cost or only give final-iterate privacy.

Approach

Combine mu^2-SGD (Anytime-GD averaging with corrected momentum) with Gaussian noise in a parallel minibatch template; the momentum makes gradient estimate errors decay as 1/t so each sample's influence shrinks, reducing the noise needed for RDP, with linear per-iteration cost.

Key results

• Theory: untrusted server rate O(1/sqrt(nM) + sqrt(d)/(eps n sqrt(M))) and trusted server rate O(1/sqrt(nM) + sqrt(d)/(eps n M)), both claimed optimal with linear computational complexity in dataset size.
• MNIST logistic regression (M=1, rho=16): loss 2.252, accuracy 70.4%; M=100 untrusted at rho=4 drops to 65.4% accuracy, trusted at M=100 gives 69.5%.
• Trusted and untrusted results converge as rho increases (less privacy), since privacy type matters less at low privacy.

Measured effects reported in the paper

Surprising findings

• Experiments show the trusted-server variant does not improve loss noticeably over untrusted at small M, and accuracy differences are small (within about 1 point at M=10).

Limitations

• Experiments are small-scale logistic regression on MNIST only, with a single noise/privacy setup.
• Convergence guarantees are shown for the homogeneous i.i.d. case only; heterogeneous analysis is not fully developed in the convergence statements.
• Trusted-shuffler setting is left as an open question.

Experimental setup

Datasets: MNIST

Benchmarks: MNIST

Metrics: test loss accuracy

Baselines: noisy minibatch SGD DP-SCO optimal population-loss methods (super-linear cost) shuffle-based DP FL method

Builds on: mu^2-SGD Anytime-GD Renyi differential privacy minibatch SGD

Closest papers by meaning

• Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation (ICML 2025 · similarity 0.94)
• Towards Trustworthy Federated Learning with Untrusted Participants (ICML 2025 · similarity 0.90)
• Faster Algorithms for User-Level Private Stochastic Convex Optimization (NeurIPS 2024 · similarity 0.89)
• Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.89)
• Federated Online Prediction from Experts with Differential Privacy: Separations and Regret Speed-ups (NeurIPS 2024 · similarity 0.89)
• Towards the Robustness of Differentially Private Federated Learning (AAAI 2024 · similarity 0.88)