Research map / Federated learning and privacy / Privacy attacks and secure learning
Privacy Attacks in Decentralized Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that honest-but-curious nodes in decentralized gradient descent (D-GD) and gossip averaging can reconstruct private data of non-neighboring nodes, via a linear-algebra attack on the gossip observations followed by gradient inversion.
Problem
Whether decentralized learning (D-GD and gossip averaging) preserves privacy against honest-but-curious nodes, specifically whether a node can reconstruct the private data of nodes outside its immediate neighborhood.
Approach
Each message an attacker receives is treated as a linear equation in the private values with coefficients given by powers of the known gossip matrix; factorizing the resulting knowledge matrix via RREF reveals which values are reconstructible. For D-GD, gradients are modeled as a fixed component plus zero-mean noise, reconstructed via generalized least squares, and then passed to an existing gradi
Key results
- In Erdos-Renyi graphs, a single attacker typically reconstructs many nodes well beyond its direct neighborhood, with the fraction increasing with connectivity and number of attackers.
- On a 31-node line graph with CIFAR-10 logistic regression, the attacker reconstructs inputs of nodes up to distance 28 from it.
- With a CNN on MNIST and a black-box gradient inversion attack on a 31-node line graph, reconstructions are accurate up to distance 26.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase graph connectivity | Gossip averaging reconstruction attack · Synthetic Erdős–Rényi graphs | fraction of reconstructed nodes | Helped |
| Increase the number of attackers | Gossip averaging reconstruction attack · Synthetic Erdős–Rényi graphs | fraction of reconstructed nodes | Helped |
| Choose an attacker with higher degree centrality | Gossip averaging reconstruction attack · Erdős–Rényi and Facebook Ego graphs | proportion of nodes reconstructed: 0.94; 0.94 | Helped |
| Increase attacker-target shortest-path length | Gossip averaging reconstruction attack · Erdős–Rényi and Facebook Ego graphs | reconstruction probability: -0.44 \pm 0.09; -0.51 \pm 0.13 | Hurt |
| Use communicability as an indicator of attacker-target relationship | Gossip averaging reconstruction attack · Erdős–Rényi and Facebook Ego graphs | correlation with reconstruction probability: 0.35 \pm 0.05; 0.08 \pm 0.35 | Mixed |
| Use smaller learning rates so gradients vary less across iterations | D-GD reconstruction attack · Florentine graph | PSNR of reconstructed images | Helped |
| Use a learning rate that is too large | D-GD reconstruction attack · Florentine graph | reconstruction accuracy | Hurt |
| Use similar local model parameters across nodes | D-GD reconstruction attack · D-GD experiments | reconstruction quality | Helped |
| Reconstruct gradients on a 31-node line graph with the attacker at an endpoint | D-GD reconstruction attack · CIFAR-10, logistic regression | maximum distance of informative reconstruction: distance 28 | Helped |
| Use black-box gradient inversion for reconstructed gradients | D-GD reconstruction attack · MNIST, CNN, 31-node line graph | maximum distance of accurate reconstruction: distance 26 | Helped |
Surprising findings
- Attackers can reconstruct data of distant nodes in D-GD even though gradients are mixed many times before reaching them.
- Which nodes are reconstructible cannot be easily characterized by simple graph properties; small graphs show counter-intuitive reconstructible sets.
Limitations
- D-GD attack relies on a gradient similarity assumption (fixed plus zero-mean noise) that is not typically satisfied in practice; it needs small learning rates.
- Assumes honest-but-curious attackers that know the graph and gossip matrix.
- Gradient-to-data step uses an existing gradient inversion attack, so reconstruction quality depends on it; the logistic regression experiments use closed-form inversion.
Experimental setup
Datasets: CIFAR-10 MNIST Facebook Ego graphs Florentine graph Erdos-Renyi random graphs line graphs random geometric graphs
Benchmarks: CIFAR-10 reconstruction on D-GD MNIST reconstruction on D-GD Facebook Ego graphs Florentine graph
Metrics: fraction of reconstructed nodes PSNR relative squared error Spearman correlation Kendall rank correlation
Baselines: Attacks on decentralized learning limited to direct neighbors
Builds on: gossip averaging Decentralized Gradient Descent gradient inversion attacks
Closest papers by meaning
- Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks (AAAI 2024 · similarity 0.89)
- The Privacy Power of Correlated Noise in Decentralized Learning (ICML 2024 · similarity 0.89)
- Differentially Private Decentralized Learning with Random Walks (ICML 2024 · similarity 0.88)
- RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent (NeurIPS 2026 · similarity 0.87)
- Revisiting Differentially Private Algorithms for Decentralized Online Learning (ICML 2025 · similarity 0.86)
- Competitive Advantage Attacks to Decentralized Federated Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.