Research map / Learning theory and algorithms / Prediction and decision losses
ByMI: Byzantine Machine Identification with False Discovery Rate Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ByMI, a p-value-free procedure that identifies Byzantine machines in distributed learning via sample-splitting symmetric score statistics and FDR control, with finite-sample guarantees and experiments on synthetic and MNIST/F-MNIST/CIFAR10 data.
Problem
Identifying Byzantine (corrupted) worker machines in distributed learning with statistically valid guarantees on false discoveries, since p-value-based tests are unreliable in high dimensions and finite samples.
Approach
Split each worker's samples into two halves, build a score W_j as the inner product of the two halves' centered gradients (symmetric around zero for normal machines), and pick a data-driven threshold from the empirical symmetry of negative vs positive scores to control FDR without p-values or dimension-dependent null distributions.
Key results
- On MNIST under OOD attack with contamination 0.1, ByMI-Filter attains FDR 6.8% and TPR 98.1% while RMDP-BH reaches FDR 88.3%.
- Under synthetic Scenario A, ByMI-based methods keep FDR near the nominal 0.1 level while Krum and FABA are overly inflated.
- Applied to distributed training under IPM attack, ByMI-Filter and ByMI-Filter+ achieve higher test accuracy than the original Filtering aggregator.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ByMI-Filter instead of RMDP-BH | Byzantine machine identification · MNIST, OOD attack | FDR (%): 88.3 -> 6.8 | Helped |
| Use ByMI-Filter instead of RMDP-BH | Byzantine machine identification · F-MNIST, OOD attack | FDR (%): 68.9 -> 7.1 | Helped |
| Use ByMI-Filter instead of RMDP-BH | Byzantine machine identification · CIFAR10, OOD attack | FDR (%): 54.1 -> 5.5 | Helped |
| Use ByMI-Filter instead of RMDP-BH | Byzantine machine identification · MNIST, IPM attack | FDR (%): 89.2 -> 6.3 | Helped |
| Use ByMI-Filter instead of RMDP-BH | Byzantine machine identification · F-MNIST, IPM attack | FDR (%): 75.3 -> 6.5 | Helped |
| Use ByMI-Filter instead of RMDP-BH | Byzantine machine identification · CIFAR10, IPM attack | FDR (%): 79.6 -> 5.4 | Helped |
| Use ByMI-Filter rather than distance-based baselines | Byzantine machine identification · OOD and IPM attacks across MNIST, F-MNIST, and CIFAR10 | FDR (%) | Helped |
| Use ByMI-Filter+ projection matrix rather than scale-matrix ByMI-Filter (ablation) | ByMI · Scenario A, p = 50, b = 0.15 | FDR (%): 9.5 -> 9.0 | No clear effect |
| Use the geometric median instead of the filtering estimator (ablation) | ByMI · Scenario A, p = 50, b = 0.15 | FDR (%): 9.5 -> 10.0 | No clear effect |
| Vary the scale estimator among the three reported choices (ablation) | ByMI-Filter · Scenario A, p = 50 and 100, b = 0.15 | FDR (%): 9.2; 9.4; 9.4 | No clear effect |
| Use dimension-dependent estimators (GEOM, MOM, or VRMOM) instead of dimension-agnostic estimators (ablation) | ByMI · Scenario A, especially p = 100 | TPR | Hurt |
| Use filtering rather than the geometric median as the robust estimator (ablation) | ByMI · Scenario B, p = 100 | TPP | Helped |
Surprising findings
- Dimension-dependent robust estimators (GEOM, MOM, VRMOM) combined with ByMI fail to control FDR in high dimensions, while dimension-agnostic filtering works better.
- ByMI works reasonably in a deep CNN setting even though the theory's assumptions on the mean-estimation error do not clearly hold, attributed to low-rank gradient structure.
Limitations
- Sample splitting halves the data used for each worker's gradient, which may sacrifice accuracy.
- Theory assumes Huber contamination model; other adversarial behaviors (e.g., transmission-path attacks) are not covered.
- FDR guarantees depend on the accuracy of the robust mean estimator; dimension-dependent estimators can break control.
Experimental setup
Datasets: MNIST Fashion MNIST CIFAR10 synthetic Gaussian data
Benchmarks: MNIST Fashion MNIST CIFAR10 synthetic mean estimation synthetic linear regression
Metrics: FDR TPR sure-detection proportion Pa FDP test accuracy estimation error
Baselines: RMDP-BH Krum FABA Zeno Filtering ByMI-GEOM
Builds on: Benjamini-Hochberg procedure Krum filtering algorithm for robust mean estimation RMDP outlier detection knockoff-style symmetry thresholding
Closest papers by meaning
- Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers (ICML 2024 · similarity 0.87)
- On the Tension between Byzantine Robustness and No-Attack Accuracy in Distributed Learning (ICML 2025 · similarity 0.86)
- Byzantine-Robust Decentralized Learning via Remove-then-Clip Aggregation (AAAI 2024 · similarity 0.85)
- Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences (NeurIPS 2024 · similarity 0.84)
- Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML (NeurIPS 2024 · similarity 0.84)
- Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.