Research map / Federated learning and privacy / Privacy attacks and secure learning

Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks

AAAI 2024 · Federated learning and privacy › Privacy attacks and secure learning · approach: Gradient inversion attacks · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Dual Gradient Pruning (DGP), which removes both the largest and smallest gradient entries with error feedback, to defend collaborative learning against passive and active gradient inversion attacks while cutting communication cost.

Problem

Gradient inversion attacks (passive optimization and analytical attacks, and active server attacks such as Rob) can recover private training data from shared gradients in collaborative learning, and existing defenses trade off privacy, utility, and efficiency poorly.

Approach

Instead of standard Top-k pruning that keeps the largest gradient entries, DGP removes the top-k1 largest entries (which enlarge gradient distance and leak label information) and the bottom-k2 smallest entries (to raise the pruning ratio and keep accuracy), combined with an error feedback mechanism.

Key results

• On CIFAR10 with ResNet18, DGP (k1+k2=80%) keeps final model accuracy at 93.40% versus 93.62% for the no-defense baseline, while achieving Rob-attack SSIM of 0.051 and LPIPS of 0.527.
• DGP reduces communication cost by about 40% per epoch relative to the baseline (e.g., ResNet18 85.25 MB to 52.22 MB).
• Under IG attack, DGP yields LPIPS 0.316 and SSIM 0.287, while Top-k (20% kept) and Outpost are vulnerable (SSIM 0.769 and 0.640 respectively).

Measured effects reported in the paper

Surprising findings

• Top-k gradient pruning, usually considered weak for privacy, performs strongly against active Rob attacks, and Outpost, which combines Top-k with noise, fails against IG and GI.
• Standard DP with sigma=1e-2 reduces final accuracy to 76.01% while failing to defend against the Rob active attack.

Limitations

• Privacy analysis is non-rigorous: Proposition relies on a first-order Taylor approximation and the authors defer a more rigorous analysis to future work.
• ADGP relies on a single randomly selected user broadcasting the location matrix; a malicious user could break the scheme.
• Privacy of ADGP against passive attacks is not evaluated.

Experimental setup

Datasets: CIFAR10 CIFAR100 ImageNet

Benchmarks: CIFAR10 CIFAR100 ImageNet

Metrics: LPIPS SSIM MSE PSNR model accuracy communication cost computation cost

Baselines: Soteria ATS Precode Outpost DP Top-k pruning CL-SGD

Builds on: Top-k gradient sparsification error feedback Inverting gradients (IG) Rob attack Outpost

Closest papers by meaning

• Distributed Learning without Distress: Privacy-Preserving Empirical Risk Minimization (NeurIPS 2018 · similarity 0.89)
• Privacy Attacks in Decentralized Learning (ICML 2024 · similarity 0.89)
• High-Fidelity Gradient Inversion in Distributed Learning (AAAI 2024 · similarity 0.88)
• The Privacy Power of Correlated Noise in Decentralized Learning (ICML 2024 · similarity 0.87)
• Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss (ICML 2024 · similarity 0.86)
• A New Federated Learning Framework Against Gradient Inversion Attacks (AAAI 2025 · similarity 0.86)