Research map / Optimization / Distributed and compressed
Bayesian Distributed Stochastic Gradient Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| predict straggler cutoff with amortized-inference generative model instead of waiting for all workers | synchronous SGD, 158 workers · MNIST 2-layer CNN, 25% workers 2x slower | throughput: naive sync loses 50% throughput; BDSGD ignores all 40 slow workers | Helped |
| use BDSGD cutoffs instead of Elfving iid-Gaussian cutoffs and naive sync | synchronous SGD, 158 workers · MNIST full training run | validation loss / wall-clock time | Helped |
| use BDSGD instead of Hogwild asynchronous SGD | 158-worker MNIST training · MNIST | wall-clock time vs final validation loss | Mixed |
| use BDSGD dynamic cutoff instead of Chen et al. fixed cutoff | synchronous SGD, 2175 workers · CIFAR10 WideResNet-16:10 production run | wall-clock speedup: 1.2% (Chen) vs 1% (BDSGD) | No clear effect |
| use BDSGD dynamic cutoff instead of Chen et al. fixed cutoff | synchronous SGD, 2175 workers · CIFAR10 ResNet-64 production run | wall-clock speedup: 9.3% (Chen) vs 7.8% (BDSGD) | No clear effect |
| use BDSGD instead of Chen et al. for cutoff prediction | synchronous SGD, 2175 workers · CIFAR10 WideResNet, ResNet-64 validation data | iteration throughput / closeness to oracle cutoff | Helped |
| refresh predicted cutoff only every ten iterations (ablation) | BDSGD · CIFAR10 ResNet-16 (small network) | wall-clock speedup: 7.6% speedup | Helped |
Closest papers by meaning
- Distributed Stochastic Optimization via Adaptive SGD (NeurIPS 2018 · similarity 0.88)
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity (NeurIPS 2026 · similarity 0.88)
- AdaGK-SGD: Adaptive Global Knowledge Guided Distributed Stochastic Gradient Descent (AAAI 2025 · similarity 0.87)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.87)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.87)
- Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglers (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.