Research map / Optimization / Distributed and compressed
On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach
Enea Monzio Compagnoni, Rustem Islamov, Frank Norbert Proske, Aurelien Lucchi, Antonio Orvieto, Eduard Gorbunov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the normalization from Thm 4.2 (Eq. 15) to DCSGD (ablation) | DCSGD with unbiased sparsification · MLP, 8 clients, affine-variance noise | gradient norm convergence | Improved |
| increase compression level omega without normalization (ablation) | DCSGD · MLP, 8 clients, affine-variance noise | gradient norm convergence | Hurt |
| use diminishing stepsize 1/sqrt(k+1) instead of constant stepsize (ablation) | DSignSGD · MLP, Student-t noise with nu=1 | gradient norm convergence | Improved |
| plain Normalized SGD instead of the prescribed normalization (ablation) | DCSGD · MLP, noise and compression | stability | Hurt |
| use first-order SDE with stability-correcting curvature drift (+eta/2 Hessian-gradient) instead of classical first/second-order SDE | SDE model of GD/SGD · quadratic and quartic toy losses | stability threshold fidelity | Improved |
Closest papers by meaning
- On Biased Compression for Distributed Learning (JMLR 2023 · similarity 0.92)
- Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits (NeurIPS 2026 · similarity 0.91)
- Unbiased Compression Saves Communication in Distributed Optimization: When and How Much? (NeurIPS 2023 · similarity 0.90)
- Towards Faster Decentralized Stochastic Optimization with Communication Compression (ICLR 2025 · similarity 0.89)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.89)
- Tight analyses of first-order methods with error feedback (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
