Research map / Optimization / Distributed and compressed

Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates

NeurIPS 2026 · Optimization › Distributed and compressed · approach: Byzantine-robust learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FAR-SIGN, a fully asynchronous, adversary-resilient signed directional-derivative method (first- and zeroth-order) with two-timescale averaging, proving a.s. convergence to stationary points and O(n^-1/4) / O(n^-1/6) rates, with MNIST wall-clock speedups over buffered robust aggregation.

Problem

Adversary-robust optimization in parameter-server/worker systems with asynchronous, stale, and possibly adversarial directional feedback; prior robust methods are synchronous or buffer updates and lack zeroth-order asynchronous robustness guarantees.

Approach

Event-driven server updates along fixed worker-specific direction dictionaries using the sign of a slow-timescale moving average of directional derivative estimates, with a directional robustness condition that bounds adversarial influence, giving a fully asynchronous method with no server reference data or buffering.

Key results

• Without attack FAR-SIGN reaches 85% accuracy in 24s versus 417s for the fastest CYBER-0 baseline (RFA), a 17.4x speedup, with max accuracy 89.6±0.4%.
• Under ALIE and Gaussian attacks FAR-SIGN reaches 85% in 70s and 35s versus 604s and 446s for the fastest baselines; overall speedups reported as 6.2–35.2x.
• CYBER-0 Trimmed Mean collapses under Gaussian and Constant attacks, reaching only 10.4% and 8.3% accuracy.

Surprising findings

• Several robust-aggregation baselines fail to reach 85% accuracy under some attacks, e.g., Trimmed Mean collapses to about 10% under Gaussian attack.

Limitations

• Assumes homogeneous worker objectives, all evaluating the same f.
• Theory assumes |W_n| = |I_n| = 1 for explicit constants; general update patterns only sketched.
• Experiments are limited to a small MLP on MNIST, not larger architectures or federated benchmarks.

Experimental setup

Datasets: MNIST

Benchmarks: MNIST

Metrics: test accuracy wall-clock time to 80% and 85% accuracy max accuracy zeroth-order calls

Baselines: CYBER-0 with buffered RFA CYBER-0 with buffered Multi-Krum CYBER-0 with buffered Median CYBER-0 with buffered Trimmed Mean CYBER-0 with buffered Krum CYBER-0 with buffered Bulyan

Builds on: signSGD CYBER-0 BASGD Robbins-Siegmund theorem

Closest papers by meaning

• On the Tension between Byzantine Robustness and No-Attack Accuracy in Distributed Learning (ICML 2025 · similarity 0.85)
• From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity (NeurIPS 2026 · similarity 0.84)
• Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML (NeurIPS 2024 · similarity 0.84)
• Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy (NeurIPS 2026 · similarity 0.84)
• When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds (NeurIPS 2026 · similarity 0.84)
• SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding (ICML 2024 · similarity 0.84)