Research map / Optimization / Distributed and compressed
Stochastic Sign Descent Methods: New Algorithms and Better Theory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase number of nodes in parallel signSGD with majority vote (ablation) | parallel signSGD (majority vote) · Rosenbrock function, d=10 | convergence rate | Helped |
| Use stochastic sign operator with momentum instead of deterministic sign in distributed setting | SSDM · distributed partitioned data | convergence rate | Helped |
| Replace stochastic sign by deterministic sign in SSDM (ablation) | SSDM · distributed, two-node counterexample | convergence | Hurt |
| Use distributed signSGD (deterministic sign of aggregated gradients) on partitioned data | signSGD in distributed setting · partitioned data, weighted losses | convergence | Hurt |
| Use mini-batch / larger batch size in signSGD training | signSGD on MNIST FNN · MNIST standard batch construction | test/train accuracy | Helped |
| Use signSGD early in training compared to SGD | signSGD vs SGD on MNIST FNN · MNIST split batch and standard batch | accuracy vs communication | Helped |
| Replace deterministic sign with stochastic sign that is unbiased after scaling | signSGD-type updates with biased estimators · counterexample (least squares, 2D) | convergence | Helped |
Closest papers by meaning
- Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction (NeurIPS 2024 · similarity 0.93)
- When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds (NeurIPS 2026 · similarity 0.90)
- z-SignFedAvg: A Unified Stochastic Sign-Based Compression for Federated Learning (AAAI 2024 · similarity 0.90)
- Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects (ICML 2025 · similarity 0.89)
- On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descent (ICLR 2025 · similarity 0.88)
- Sign-Based Optimizers Are Effective Under Heavy-Tailed Noise (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.