Research map / Optimization / Optimizers
GraB: Finding Provably Better Data Permutations than Random Reshuffling
Yucheng Lu, Wentao Guo, Christopher De Sa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GraB's online gradient balancing to choose and update example orderings instead of random reshuffling | SGD · MNIST, CIFAR10, WikiText-2, GLUE | training and validation loss | Improved |
| Use GraB's online gradient balancing to choose and update example orderings instead of random reshuffling | SGD · MNIST, CIFAR10, WikiText-2, GLUE | validation performance | Improved |
| Replace random reshuffling with Shuffle Once | SGD · MNIST, CIFAR10, WikiText-2, GLUE | convergence | Hurt |
| Replace random reshuffling with FlipFlop | SGD · MNIST, CIFAR10, WikiText-2, GLUE | convergence | No clear effect |
| Replace GraB's evolving order with a fixed order learned in the first epoch (ablation) | GraB · MNIST, CIFAR10 | training loss | Hurt |
| Use the final-epoch order from a full GraB run as a fixed order in a new run (ablation) | GraB · MNIST | training and validation performance | No clear effect |
| Use the final-epoch order from a full GraB run as a fixed order in a new run (ablation) | GraB · CIFAR10 | training and validation performance | Hurt |
| Use one learned fixed permutation rather than continually improving the permutation (ablation) | GraB · MNIST, CIFAR10 | performance relative to RR and SO | Mixed |
Closest papers by meaning
- Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization (NeurIPS 2024 · similarity 0.86)
- Are Greedy Task Orderings Better Than Random in Continual Linear Regression? (NeurIPS 2025 · similarity 0.86)
- Don't Compress Gradients in Random Reshuffling: Compress Gradient Differences (NeurIPS 2024 · similarity 0.86)
- Uncertainty Herding: One Active Learning Method for All Label Budgets (ICLR 2025 · similarity 0.85)
- Incremental Gradient Descent with Small Epoch Counts is Surprisingly Slow on Ill-Conditioned Problems (ICML 2025 · similarity 0.84)
- Stochastic Optimization with Arbitrary Recurrent Data Sampling (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
