Research map / Federated learning and privacy / Privacy attacks and secure learning
A Huber Loss Minimization Approach to Byzantine Robust Federated Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Byzantine-robust federated learning aggregator that minimizes a multi-dimensional Huber loss over client gradients, with theory under i.i.d., unbalanced, and mildly heterogeneous data and experiments on synthetic linear regression and MNIST.
Problem
Byzantine-robust gradient aggregation in federated learning, where an unknown fraction of clients send arbitrary gradients, with unbalanced client data sizes and heterogeneous clients.
Approach
Aggregate client gradients by minimizing a sample-size-weighted, vector-generalized Huber loss, with per-client thresholds T_i chosen adaptively (smaller for larger n_i). This gives near-optimal dependence on the attack ratio epsilon without knowing epsilon, and an iteratively reweighted (Weiszfeld-style) solver.
Key results
- On synthetic linear regression with epsilon=0.2 the Huber aggregator remains robust under sign-flip, Krum Attack, Trimmed Mean Attack and its own HLMA attack, while Krum fails under KA.
- On MNIST with epsilon=0.4 the method keeps good accuracy while coordinate-wise trimmed mean and median degrade notably under TMA and HLMA.
- Theory claims error rate epsilon/sqrt(n)+1/sqrt(N) (up to log factors), nearly minimax optimal in epsilon, and an epsilon-agnostic choice of T.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Huber loss minimization aggregation instead of the compared aggregators | Byzantine-robust federated learning · Synthesized linear regression, ε=0.2; sign-flip, KA, TMA, and HLMA | square root of ℓ2 regression loss | Helped |
| use Huber loss minimization aggregation instead of the compared aggregators | Byzantine-robust federated learning · MNIST, ε=0.2; sign-flip, KA, TMA, and HLMA | test accuracy | Helped |
| use Huber loss minimization aggregation instead of coordinate-wise median or trimmed mean | Byzantine-robust federated learning · MNIST, ε=0.2; sign-flip, KA, TMA, and HLMA | test accuracy | Helped |
| use Huber loss minimization aggregation instead of Krum | Byzantine-robust federated learning · MNIST, ε=0.2; Krum Attack | test accuracy | Helped |
| use Huber loss minimization aggregation instead of coordinate-wise aggregation | Byzantine-robust federated learning · MNIST, ε=0.4; TMA and HLMA | test accuracy | Helped |
| use Huber loss minimization aggregation instead of Krum | Byzantine-robust federated learning · MNIST, ε=0.4; Krum Attack | test accuracy | Helped |
| use Huber loss minimization aggregation instead of geometric median-of-means | Byzantine-robust federated learning · MNIST, ε=0.4 | test accuracy | Mixed |
| adapt Huber thresholds to client sample sizes for unbalanced allocation | Huber loss minimization · Synthesized data and MNIST, ε=0.2; HLMA | square root of ℓ2 regression loss / test accuracy | Helped |
| adapt Huber thresholds to client sample sizes for unbalanced allocation | Huber loss minimization · Synthesized data and MNIST, ε=0.2; HLMA | square root of ℓ2 regression loss / test accuracy | No clear effect |
| use unbalanced rather than balanced sample allocation | Compared federated learning methods · Synthesized data and MNIST, ε=0.2; HLMA | square root of ℓ2 regression loss / test accuracy | Hurt |
| use Huber loss minimization aggregation instead of existing methods | Byzantine-robust federated learning · Non-i.i.d. linear model, ε=0.2; KA and TMA | square root of ℓ2 regression loss | Helped |
Surprising findings
- Under unbalanced client sizes with adaptive thresholds T_i, the error rate is claimed to match the balanced case.
Limitations
- Dependence on dimension d is not optimal, as the authors acknowledge.
- Theory relies on i.i.d. or only mildly heterogeneous assumptions, and sub-exponential gradient tails.
- Experiments are small (linear regression and one-hidden-layer MNIST net) and baselines are tuned with exact epsilon knowledge.
Experimental setup
Datasets: MNIST synthetic linear regression data
Benchmarks: MNIST synthetic linear regression
Metrics: l2 regression loss (square root) test accuracy
Baselines: Krum geometric median-of-means coordinate-wise median coordinate-wise trimmed mean
Builds on: Weiszfeld's algorithm geometric median-of-means coordinate-wise trimmed mean Krum
Closest papers by meaning
- Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradients (NeurIPS 2025 · similarity 0.91)
- Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning (ICML 2024 · similarity 0.88)
- Rethinking Byzantine Robustness in Federated Recommendation from Sparse Aggregation Perspective (AAAI 2025 · similarity 0.87)
- Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates (ICML 2024 · similarity 0.87)
- From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity (NeurIPS 2026 · similarity 0.87)
- On the Tension between Byzantine Robustness and No-Attack Accuracy in Distributed Learning (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.