Research map / Federated learning and privacy / Communication-efficient FL
Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fed-NGA, a federated aggregation rule that averages normalized client gradients, giving O(pM) aggregation cost while claiming Byzantine robustness and convergence under non-IID data.
Problem
Byzantine-robust federated learning under data heterogeneity, where existing robust aggregators have high computational cost on the server.
Approach
Normalize each uploaded client vector to unit norm before taking the weighted mean, so that aggregation costs O(pM) and outlier magnitudes from Byzantine clients are bounded; convergence is proven under Lipschitz, bounded-noise, and bounded-heterogeneity assumptions, with a zero-optimality-gap result under type II (directional) assumptions.
Key results
- On TinyImageNet with no attack, Fed-NGA reaches 56.95% max test accuracy versus 55.68% for GM and 48.45% for Median.
- Under Same-value attack on MNIST (MLP, beta=0.6), Fed-NGA reaches 83.66% while GM reaches 70.11% and Median 61.60%.
- Aggregation on CIFAR10 with LeNet is reported as at least 100x faster than the other robust baselines (e.g., Fed-NGA 0.0558 s vs Median 22.16 s without attack).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use normalized-gradient weighted-mean aggregation instead of FedAvg | Fed-NGA · TinyImageNet, no attack, β=0.6 | test accuracy (%): 55.67 -> 56.95 | Helped |
| Use normalized-gradient weighted-mean aggregation instead of robust baselines | Fed-NGA · TinyImageNet, all six attack conditions, β=0.6 | test accuracy (%) | Helped |
| Use normalized-gradient weighted-mean aggregation instead of robust baselines | Fed-NGA · CIFAR10, β=0.6, attacks and no attack | test accuracy (%) | Helped |
| Use normalized-gradient weighted-mean aggregation instead of robust baselines | Fed-NGA · MNIST, Same-value and FoE attacks, β=0.6 | test accuracy (%) | Helped |
| Use normalized-gradient weighted-mean aggregation instead of the strongest baseline | Fed-NGA · MNIST, LIE attack, β=0.6 | test accuracy (%): 94.93 vs 94.92 | No clear effect |
| Normalize each uploaded client vector before weighted-mean aggregation | Fed-NGA · Aggregation runtime, TinyImageNet, CIFAR10, and MNIST | running time | Helped |
| Decrease the Dirichlet concentration parameter β, increasing data heterogeneity | Fed-NGA · TinyImageNet | test accuracy (%) | Hurt |
Surprising findings
- FedAvg collapses to about 10% accuracy under several attacks (e.g., Gaussian and Same-value on CIFAR10), while Fed-NGA remains near its clean accuracy.
- Under Sign-flip and FoE attacks, the MCA and CClip running times rise to over 1000 s on CIFAR10 due to iterative computation.
Limitations
- Zero optimality gap requires stronger conditions (type II assumptions or small inner error and heterogeneity throughout training), and the authors note more general assumptions may be needed.
- Fed-NGA requires different learning rate schedules than baselines, so the comparison depends on per-method tuning.
- Experiments are limited to small models and three image datasets, with 50 clients and 100-500 iterations.
Experimental setup
Datasets: MNIST CIFAR10 TinyImageNet
Benchmarks: MNIST CIFAR10 TinyImageNet
Metrics: maximum test accuracy aggregation running time
Baselines: FedAvg Median Krum Geometric Median MCA CClip
Builds on: FedAvg geometric median centered clipping maximum correntropy aggregation
Closest papers by meaning
- A Huber Loss Minimization Approach to Byzantine Robust Federated Learning (AAAI 2024 · similarity 0.91)
- Self-Driven Entropy Aggregation for Byzantine-Robust Heterogeneous Federated Learning (ICML 2024 · similarity 0.90)
- Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates (ICML 2024 · similarity 0.89)
- From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity (NeurIPS 2026 · similarity 0.89)
- Exploit Gradient Skewness to Circumvent Byzantine Defenses for Federated Learning (AAAI 2025 · similarity 0.88)
- FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.