Research map / Optimization / Nonconvex and smooth optimization
Random Reshuffling is Not Always Better
Christopher M De Sa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| switch from with-replacement to random reshuffling (parallel SGD with averaging) | Parallel SGD (Algorithm 1) · Matrix-completion-like task, n=40, M=1000 | loss gap over 100 epochs | Hurt |
| use random reshuffling instead of with-replacement with the same small step size | Parallel SGD · Noiseless convex quadratics | distance to optimum | Improved |
| use random reshuffling with separately tuned best step sizes | Parallel SGD, M=100 · Convex task n=8, K=20 | distance to optimum | Hurt |
| remove averaging (M=1, plain SGD) (ablation) | Parallel SGD · Convex task n=8 | distance to optimum | Mixed |
| use random reshuffling in plain SGD without averaging | SGD, step size 1/2 · Constructed 3-matrix plus R task, 20 runs, 1000 epochs | loss | Hurt |
Closest papers by meaning
- On the Last-Iterate Convergence of Shuffling Gradient Methods (ICML 2024 · similarity 0.86)
- Improved Last-Iterate Convergence of Shuffling Gradient Methods for Nonsmooth Convex Optimization (ICML 2025 · similarity 0.86)
- Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences (NeurIPS 2024 · similarity 0.86)
- Provable Benefit of Random Permutations over Uniform Sampling in Stochastic Coordinate Descent (ICML 2025 · similarity 0.85)
- Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization (NeurIPS 2024 · similarity 0.85)
- Revisiting Convergence: Shuffling Complexity Beyond Lipschitz Smoothness (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
