Research map / Federated learning and privacy / Differential privacy
The Privacy Power of Correlated Noise in Decentralized Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DECOR, a decentralized SGD variant that injects pairwise-cancelling correlated Gaussian noise (from shared seeds) plus uncorrelated noise, and proves it matches the central-DP privacy-utility trade-off under a new relaxation SecLDP.
Problem
Decentralized learning with differential privacy against external eavesdroppers and honest-but-curious users; local DP incurs an n-fold worse privacy-utility trade-off than central DP, and prior network-DP relaxations only match CDP for well-connected graphs.
Approach
Users share randomness seeds with neighbors in one round, generate pairwise-cancelling correlated Gaussian noise injected before gossip averaging, plus uncorrelated noise after; the residual correlated noise on sparse graphs is controlled in convergence analysis, yielding SecLDP guarantees and CDP-matching utility on arbitrary connected graphs.
Key results
- DECOR matches the CDP baseline and achieves roughly an order of magnitude lower training loss than the LDP baseline on least-squares and logistic regression across privacy levels and topologies (ring, grid, fully connected, n=16).
- On MNIST with a one-hidden-layer network, the CDP–LDP accuracy gap is almost 10 points at the lowest privacy budget, while DECOR on the ring or grid is within 1 accuracy point of CDP.
- Theory: each DECOR step satisfies (α, αε)-SecRDP with ε ≤ 2C²(1/(nσ_dp²) + (1−1/n)/(σ_dp²+a(G)σ_cor²)), and the trade-off O(C²d log(1/δ)/(n²ε²)) matches the central DP lower bound on arbitrary connected graphs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Decor instead of the LDP baseline | Decentralized learning · Synthetic least-squares regression | training loss: an order of magnitude lower | Helped |
| Use Decor instead of the LDP baseline | Decentralized learning · a9a logistic regression | training loss: an order of magnitude better | Helped |
| Use Decor instead of the CDP baseline | Decentralized learning · a9a logistic regression | training loss | No clear effect |
| Use Decor instead of the LDP baseline | Decentralized learning · MNIST | accuracy: almost 10 accuracy points | Helped |
| Use Decor on the ring topology instead of the CDP baseline | Decentralized learning · MNIST | accuracy: less than 1 accuracy point | No clear effect |
Limitations
- Matching CDP requires stronger connectivity assumptions (2-connected graph) for honest-but-curious non-colluding users; a curious neighbor can cancel correlated noise on a non-2-connected graph.
- Guarantees weaken as collusion grows; for large colluding fractions SecLDP approaches LDP.
- Assumes shared secrets (randomness seeds) exchanged securely in one round; convergence has a slowdown term from non-cancelled correlated noise proportional to the mixing heterogeneity.
Experimental setup
Datasets: MNIST a9a synthetic least-squares regression data
Benchmarks: MNIST a9a synthetic least-squares regression
Metrics: training loss test accuracy
Baselines: LDP baseline (D-SGD with local Gaussian noise) CDP baseline (D-SGD with central Gaussian noise) Network DP relaxation
Builds on: decentralized SGD local differential privacy correlated Gaussian noise for decentralized averaging Rényi differential privacy network differential privacy
Closest papers by meaning
- Differentially Private Decentralized Learning with Random Walks (ICML 2024 · similarity 0.91)
- Revisiting Differentially Private Algorithms for Decentralized Online Learning (ICML 2025 · similarity 0.90)
- Mitigating the Privacy–Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy (NeurIPS 2025 · similarity 0.90)
- Privacy Attacks in Decentralized Learning (ICML 2024 · similarity 0.89)
- Noisy SIGNSGD Is More Differentially Private Than You (Might) Think (ICML 2025 · similarity 0.88)
- Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.