Research map / Adversarial robustness and security / Attacks on text and graphs
SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes signSGD-FD, which decodes gradient signs with LLR-weighted majority voting whose weights are estimated online, so that adversarial sign-flipping workers' signals are exploited rather than discarded, keeping convergence unaffected when adversaries are fewer than benign workers.
Problem
Distributed signSGD with majority voting loses convergence rate and test accuracy as the number of adversarial workers grows under sign-flip and sign-inversion attacks.
Approach
Models the sign-communication channel as parallel binary symmetric channels, derives ML decoding as LLR-weighted majority voting, and estimates per-worker crossover probabilities from past decoding results so that adversarial workers (crossover above 1/2) contribute with negative weights rather than being removed.
Key results
- Under sign-inversion attack with L=6 of M=15 workers, signSGD-FD reaches near no-attack accuracy while signSGD-MV deteriorates significantly as L grows.
- signSGD-FD fails to converge at L=9 (more than half of workers attacked), consistent with the theory's threshold of L < M/2.
- Communication comparison: signSGD-type methods reduce communication by about 30x versus Multi-Krum on SGD, and Top-K SGD with Multi-Krum needs about 10x more bits than signSGD-FD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use signSGD-FD instead of signSGD-MV | signSGD-style training · SIA (r=1), MNIST and CIFAR-10, L < M/2 | test accuracy | Helped |
| increase the number of compromised workers using signSGD-MV | signSGD-MV · SIA (r=1), MNIST and CIFAR-10 | test accuracy | Hurt |
| use signSGD-FD instead of signSGD-MV | signSGD-style training · SSFA (r = 1/2), MNIST and CIFAR-10 | test accuracy | Helped |
| increase the sign-flipping probability r | signSGD-FD and signSGD-MV · SSFA, MNIST and CIFAR-10, L=6 | test accuracy | Hurt |
| set the sign-flipping probability to r=0 or 1 rather than r=1/2 | signSGD-FD · SSFA, MNIST and CIFAR-10, L=6 | test accuracy | Helped |
| use signSGD-FD instead of other signSGD-style robust optimizers | signSGD-style robust optimizers · SIA (r=1), L=6 | test accuracy | Helped |
| use signSGD-FD instead of full-precision Multi-Krum algorithms | attack-robust distributed learning · MNIST, Gaussian Byzantine attack | communication cost reduction: 30x | Helped |
| use Top-K SGD-based Multi-Krum instead of signSGD-FD | attack-robust distributed learning · MNIST, Gaussian Byzantine attack | communication cost: 10x more costs | Hurt |
| use MV rather than WMV during signSGD-FD's initial phase (ablation) | signSGD-FD · CIFAR-10, SIA (r=1), L=6 | test accuracy | Hurt |
| vary the initial phase duration T_in, except when too short to collect error samples (ablation) | signSGD-FD · CIFAR-10, SIA (r=1), L=6 | test accuracy | No clear effect |
Surprising findings
- Adversarial workers with sign-inversion attacks can be exploited: their gradient signs are kept with negative LLR weights, so the convergence rate is invariant to the number of attackers as long as they are fewer than benign workers.
- Under the stochastic sign-flip attack with r=1/2 (uniform random flipping), the worst-case degradation occurs, not the sign-inversion case.
Limitations
- Convergence guarantee holds only when adversaries are fewer than benign workers; signSGD-FD fails to converge at L=9 of 15.
- Experiments are limited to small image classification models (CNN, ResNet-56) on MNIST and CIFAR, with IID data across workers.
- Theoretical analysis relies on assumptions including known-ish error estimates and bounded LLR weight estimation ratio.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100
Benchmarks: MNIST CIFAR-10 CIFAR-100
Metrics: test accuracy convergence rate communication cost (bits)
Baselines: signSGD-MV Election-signSGD Sto-signSGD Noisy signSGD Distributed SGD with Multi-Krum Top-K SGD with Multi-Krum
Builds on: signSGD-MV Multi-Krum Sto-signSGD
Closest papers by meaning
- signSGD with Majority Vote is Communication Efficient and Fault Tolerant (ICLR 2019 · similarity 0.91)
- Noisy SIGNSGD Is More Differentially Private Than You (Might) Think (ICML 2025 · similarity 0.85)
- Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction (NeurIPS 2024 · similarity 0.85)
- On the Convergence of an Adaptive Momentum Method for Adversarial Attacks (AAAI 2024 · similarity 0.85)
- z-SignFedAvg: A Unified Stochastic Sign-Based Compression for Federated Learning (AAAI 2024 · similarity 0.85)
- RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.