AI Paper Atlas

Research map / Optimization / Optimizers

Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies

Oscar Li, James Harrison, Jascha Sohl-Dickstein, Virginia Smith, Luke Metz

NeurIPS 2023 · Optimization › Optimizers · approach: Randomized gradient estimation · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
noise-sharing period K = T (NRES) instead of smaller K (ablation)GPES_K · learned optimizer, T=1000total gradient varianceImproved
noise-sharing period K = T (NRES) instead of smaller K (ablation)GPES_K · learned optimizer meta-trainingconvergence speedImproved
noise-sharing period K = T (NRES) instead of smaller K (ablation)GPES_K · Lorenz systemoptimization trajectory stabilityImproved
replace PES by NRESPES · learned optimizerwall-clock time to target loss: 5x longer for PESImproved
replace FullES by NRESFullES · learned optimizerwall-clock time to target loss: 9x longer for FullESImproved
replace TES by NRESTES · learned optimizerreaching NRES target lossImproved
replace PES by NRESPES · Lorenz systemconvergenceImproved
replace FullES by NRESFullES · Lorenz systemloss vs wall-clock timeImproved
replace TES by NRESTES · Mujoco Swimmer and Half Cheetahreward vs sequential stepsImproved
replace PES by NRESPES · Mujoco Swimmer and Half Cheetahsequential steps to solveImproved
replace FullES by NRESFullES · Mujoco Swimmer and Half Cheetahsequential steps with perfect parallelization: 10x and 60x longerImproved
average T/W iid NRES estimates instead of one FullES estimateFullES · learned optimizertotal gradient variance: 1000 times wall-clock speed upImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy