Research map / Optimization / Optimizers
Using Statistics to Automate Stochastic Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a Markov-chain statistical test that accounts for variance in the stationarity statistic instead of Yaida's deterministic ratio test (ablation) | SASA on MNIST logistic regression · MNIST | test accuracy: 93.84% -> 94.08% | Helped |
| Account for correlation in stationarity samples using batch-means variance estimation rather than an i.i.d. variance estimate (ablation) | SASA · CIFAR-10 and MNIST | robustness of learning-rate schedule | Helped |
| Use Yaida's deterministic ratio test without accounting for statistic variance (ablation) | SASA on small-batch CIFAR-10 · CIFAR-10 | robustness of learning-rate schedule | Hurt |
| Vary the SASA drop factor while allowing the learning rate to decay to the same level (ablation) | SASA on CIFAR-10 and ImageNet · CIFAR-10 and ImageNet | final performance | No clear effect |
| Increase the equivalence threshold δ (ablation) | SASA · CIFAR-10, ImageNet, and MNIST | learning-rate drop timing | Mixed |
| Increase the confidence parameter γ (ablation) | SASA · CIFAR-10, ImageNet, and MNIST | learning-rate drop timing | Mixed |
| Use SASA instead of hand-tuned constant-and-cut SGM | Mask R-CNN ResNet50 FPN · COCO | bbox-AP | Helped |
| Use SASA instead of hand-tuned constant-and-cut SGM | Mask R-CNN ResNet50 FPN · COCO | training iterations to comparable performance: 200000 vs 90000 | Hurt |
Closest papers by meaning
- Do Stochastic, Feel Noiseless: Stable Stochastic Optimization via a Double Momentum Mechanism (ICLR 2025 · similarity 0.90)
- Asymptotic theory of SGD with a general learning-rate (NeurIPS 2025 · similarity 0.90)
- A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance (ICML 2021 · similarity 0.89)
- A Variational Analysis of Stochastic Gradient Algorithms (ICML 2016 · similarity 0.89)
- Learning Rate Annealing Improves Tuning Robustness in Stochastic Optimization (ICML 2026 · similarity 0.89)
- Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.