Research map / Federated learning and privacy / Communication-efficient FL
Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends differentially private mu^2-SGD federated learning to partial participation via a noise-cancellation mechanism with participation-dependent noise, achieving optimal excess loss with linear computation in the untrusted-server setting.
Problem
Differentially private federated stochastic convex optimization under partial participation, where only a subset of machines communicate each round; prior optimal methods only handled full participation or had super-linear cost.
Approach
Each machine injects fresh noise when it participates and cancels its previous noise on the next participation (correlated noise telescoping), with per-machine noise variance scaled by its number of participations, so the server's aggregated noise stays small and the optimal rate holds under untrusted server.
Key results
- On MNIST logistic regression (M=100, m=50, rho=8), the method reaches 63.7% accuracy in 13 s versus 58.9% for Noisy SGD (9 s) and 63.3% for Other Work (282 s).
- Theory: excess loss O(1/sqrt(n) + sqrt(Md)/(epsilon n)) under untrusted server with partial participation, matching the lower bound with linear computation.
- At rho=12 Other Work (66.7%) slightly outperforms the proposed method (66.5%), but at 730 s versus 13 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase privacy parameter ρ from 4 to 12 | Our Work · MNIST test | accuracy: 53.8% -> 66.5% | Helped |
| Increase privacy parameter ρ from 4 to 12 | Noisy SGD · MNIST test | accuracy: 45.1% -> 63.7% | Helped |
| Increase privacy parameter ρ from 4 to 12 | Other Work · MNIST test | accuracy: 47.6% -> 66.7% | Helped |
| Increase participating machines m from 20 to 80 | Our Work · MNIST test | accuracy: 60.8% -> 63.8% | Helped |
| Increase participating machines m from 20 to 80 | Other Work · MNIST test | accuracy: 59.7% -> 65.8% | Helped |
| Increase participating machines m from 50 to 80 | Noisy SGD · MNIST test | accuracy: 58.9% -> 57.0% | Hurt |
| Use Our Work instead of Noisy SGD at ρ=8 | MNIST logistic regression · MNIST test | accuracy: 58.9% -> 63.7% | Helped |
| Use Our Work instead of Other Work at ρ=8 | MNIST logistic regression · MNIST test | time: 282 sec -> 13 sec | Helped |
Surprising findings
- At the highest privacy-budget setting (rho=12), the Other Work baseline slightly beats the proposed method in accuracy, though at far higher running time.
Limitations
- Experiments are limited to a single MNIST logistic regression setting.
- Assumes a fixed number of participating machines per round chosen uniformly at random.
- Noise scaling uses an upper bound on gradient norm S = G + 2LD, which may be loose in practice.
Experimental setup
Datasets: MNIST
Benchmarks: MNIST
Metrics: test accuracy running time
Baselines: Noisy SGD Other Work (large-batch partial-participation DP method)
Builds on: mu^2-SGD DP-mu^2-FL (full participation) STORM zero-concentrated differential privacy
Closest papers by meaning
- Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems (ICML 2024 · similarity 0.94)
- Convergent Differential Privacy Analysis for General Federated Learning (ICLR 2026 · similarity 0.91)
- Clients Collaborate: Flexible Differentially Private Federated Learning with Guaranteed Improvement of Utility-Privacy Trade-off (ICML 2025 · similarity 0.90)
- Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.89)
- Towards Trustworthy Federated Learning with Untrusted Participants (ICML 2025 · similarity 0.89)
- Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.