Research map / Optimization / Optimizers
Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies
Oscar Li, James Harrison, Jascha Sohl-Dickstein, Virginia Smith, Luke Metz
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| noise-sharing period K = T (NRES) instead of smaller K (ablation) | GPES_K · learned optimizer, T=1000 | total gradient variance | Improved |
| noise-sharing period K = T (NRES) instead of smaller K (ablation) | GPES_K · learned optimizer meta-training | convergence speed | Improved |
| noise-sharing period K = T (NRES) instead of smaller K (ablation) | GPES_K · Lorenz system | optimization trajectory stability | Improved |
| replace PES by NRES | PES · learned optimizer | wall-clock time to target loss: 5x longer for PES | Improved |
| replace FullES by NRES | FullES · learned optimizer | wall-clock time to target loss: 9x longer for FullES | Improved |
| replace TES by NRES | TES · learned optimizer | reaching NRES target loss | Improved |
| replace PES by NRES | PES · Lorenz system | convergence | Improved |
| replace FullES by NRES | FullES · Lorenz system | loss vs wall-clock time | Improved |
| replace TES by NRES | TES · Mujoco Swimmer and Half Cheetah | reward vs sequential steps | Improved |
| replace PES by NRES | PES · Mujoco Swimmer and Half Cheetah | sequential steps to solve | Improved |
| replace FullES by NRES | FullES · Mujoco Swimmer and Half Cheetah | sequential steps with perfect parallelization: 10x and 60x longer | Improved |
| average T/W iid NRES estimates instead of one FullES estimate | FullES · learned optimizer | total gradient variance: 1000 times wall-clock speed up | Improved |
Closest papers by meaning
- Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation (NeurIPS 2024 · similarity 0.87)
- Diffusion Models are Evolutionary Algorithms (ICLR 2025 · similarity 0.86)
- Automatic Differentiation of Optimization Algorithms with Time-Varying Updates (ICML 2025 · similarity 0.86)
- Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent (ICML 2025 · similarity 0.85)
- On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization (NeurIPS 2025 · similarity 0.85)
- Gradient-Variation Online Learning under Generalized Smoothness (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
