Research map / Federated learning and privacy / Differential privacy

Noisy SIGNSGD Is More Differentially Private Than You (Might) Think

ICML 2025 · Federated learning and privacy › Differential privacy · approach: Private training with public data

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that the sign compressor amplifies privacy of Noisy SIGNSGD (f-DP analysis, tight in the scalar case), that Gaussian noise beats logistic noise in estimation error, and establishes convergence of Noisy SIGNSGD with averaging and majority-vote aggregation.

Problem

Noisy SIGNSGD for communication-efficient distributed learning under data heterogeneity was believed to have the same privacy as DP-SGD, and it was unclear how the information loss from taking signs affects its privacy and estimation error versus logistic noise.

Approach

Quantifies the privacy amplification from applying sign() after noise addition using f-differential privacy, derives a tight f-DP tradeoff for the scalar generic NoisySign compressor, extends to vectors via a Gaussian-limit argument, and shows Gaussian noise yields smaller sign-estimation error than logistic noise at equal privacy.

Key results

• On Fashion-MNIST at mu=1.6 per round, G-NoisySign reaches 79.57% vs 79.90% for Gaussian-noise DP-SGD, and L-NoisySign reaches 79.66%.
• On CIFAR-10 at mu=16, G-NoisySign reaches 66.82% vs 67.53% for Gaussian noise DP-SGD.
• Theory shows G-NoisySign's privacy limit as d grows is mu = 2C/sigma * sqrt(pi/2), a sqrt(pi/2) improvement over the Gaussian mechanism, and G-NoisySign has smaller estimation error than L-NoisySign in the high-privacy regime.

Measured effects reported in the paper

Surprising findings

• Majority-vote aggregation at the server costs less than 2% accuracy in most settings while the stated central privacy should improve.
• The privacy amplification claimed in prior logistic-noise work (1.5x more steps) is mainly due to ignoring the sign compressor's amplification, per the authors' analysis.

Limitations

• Gradient bound assumption is used in place of clipping; the effect of clipping is left for future work.
• Privacy amplification of the majority-vote server aggregation is not analyzed.
• Privacy analysis is local DP, and the vector-case bound holds only in the high-dimension limit.

Experimental setup

Datasets: Fashion-MNIST CIFAR-10

Benchmarks: Fashion-MNIST CIFAR-10

Metrics: test accuracy f-DP tradeoff function probability of wrong aggregation estimation error

Baselines: DP-SGD with Gaussian noise L-NoisySign G-NoisySign majority vote L-NoisySign majority vote

Builds on: SIGNSGD DP-SGD Noisy SIGNSGD (Chen et al., 2020b) logistic mechanism (Jang et al., 2024) f-DP

Closest papers by meaning

• z-SignFedAvg: A Unified Stochastic Sign-Based Compression for Federated Learning (AAAI 2024 · similarity 0.91)
• Rethinking DP-SGD in Discrete Domain: Exploring Logistic Distribution in the Realm of signSGD (ICML 2024 · similarity 0.90)
• cpSGD: Communication-efficient and differentially-private distributed SGD (NeurIPS 2018 · similarity 0.89)
• Sketched Gaussian Mechanism for Private Federated Learning (NeurIPS 2025 · similarity 0.89)
• The Privacy Power of Correlated Noise in Decentralized Learning (ICML 2024 · similarity 0.88)
• Privacy Amplification by Iteration for ADMM with (Strongly) Convex Objective Functions (AAAI 2024 · similarity 0.86)