Research map / Optimization / Distributed and compressed
Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine resilient averaging with distributed momentum (β = 0.99) | RESAM · MNIST and CIFAR-10 under Byzantine attacks | top-1 cross-accuracy | Helped |
| Remove distributed momentum (β = 0) (ablation) | RESAM with a resilient averaging rule · MNIST and CIFAR-10, including the little attack | top-1 cross-accuracy | Hurt |
| Use momentum without resilient averaging (ablation) | CGE · CIFAR-10 under Byzantine attacks | top-1 cross-accuracy | No clear effect |
| Set momentum to β = 0.99 (ablation) | CC · Byzantine attacks, little and label-flipping | top-1 cross-accuracy | Mixed |
| Use an aggregation rule with a smaller resilience coefficient λ | RESAM · Byzantine distributed SGD | convergence rate | Helped |
| Use MDA, CWTM, or MeaMed rather than Krum*, GM, or CWMed | RESAM with few Byzantine workers (f ≪ n) · Byzantine distributed SGD | convergence rate: MDA, CWTM, and MeaMed: \mathcal{O}\left( \nicefrac{\sigma}{\sqrt{n T}} \right); Krum^*, GM and CWMed: \mathcal{O}\left( | Helped |
| Increase the number of Byzantine workers | RESAM with aggregation rules other than CWMed · Byzantine distributed SGD | convergence rate | Hurt |
Closest papers by meaning
- Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers (AAAI 2024 · similarity 0.82)
- Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers (ICML 2024 · similarity 0.82)
- Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training (ICML 2024 · similarity 0.81)
- Resilient Byzantine Agreement with Predictions (NeurIPS 2026 · similarity 0.81)
- Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences (NeurIPS 2024 · similarity 0.81)
- Byzantine-Robust Decentralized Learning via Remove-then-Clip Aggregation (AAAI 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.