Research map / Optimization / Distributed and compressed

RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent

NeurIPS 2026 · Optimization › Distributed and compressed · approach: Byzantine-robust learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RESIST, a decentralized consensus gradient descent algorithm with CWTM screening that is robust to time-varying link-level man-in-the-middle attacks, with geometric/sublinear convergence and statistical learning-rate guarantees.

Problem

Decentralized empirical risk minimization where adversaries perform man-in-the-middle attacks on communication links, altering transmitted model vectors, which can cause models to deviate from ERM solutions.

Approach

A multistep consensus gradient descent where each local gradient step is preceded by J-1 coordinate-wise trimmed mean communication/filtering rounds, analyzed under a time-varying link-level MITM attack model (rather than node-level Byzantine), with convergence guarantees for strongly convex, PL, and smooth nonconvex ERM and statistical learning rates.

Key results

• Experiments on MNIST show RESIST achieves accuracy comparable to other algorithms across diverse system and algorithm settings for strongly convex losses.
• On CIFAR-10 nonconvex objectives, RESIST shows strong performance and robustness across attack strategies, screening methods, and loss functions.
• J-ablation experiments indicate larger J can improve convergence and reduce local gradient updates at a fixed communication budget.

Limitations

• Heterogeneous local objectives yield only residual-neighborhood guarantees rather than exact convergence.
• Communication and filtering costs still scale with model dimension; CWTM requires sorting per coordinate.
• Non-i.i.d. statistical learning-rate guarantees are left to future work.

Experimental setup

Datasets: MNIST CIFAR-10

Benchmarks: MNIST CIFAR-10

Metrics: accuracy

Baselines: BRIDGE-T RedGraf-CWTM DGD

Builds on: BRIDGE ByRDiE RedGraf decentralized gradient descent coordinate-wise trimmed mean

Closest papers by meaning

• From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity (NeurIPS 2026 · similarity 0.88)
• Byzantine-Robust Decentralized Learning via Remove-then-Clip Aggregation (AAAI 2024 · similarity 0.88)
• Privacy Attacks in Decentralized Learning (ICML 2024 · similarity 0.87)
• Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences (NeurIPS 2024 · similarity 0.87)
• Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training (ICML 2024 · similarity 0.86)
• On the Tension between Byzantine Robustness and No-Attack Accuracy in Distributed Learning (ICML 2025 · similarity 0.86)