AI Paper Atlas

Research map / Optimization / Optimizers

Romberg-Extrapolated Zeroth-Order Gradient Estimator: Higher-Order Bias Reduction with Preserved Leading Directional Variance

Hongcheng Dong, Wenqiang Pu, Licheng Zhao, Rui Zhou, Feng Yin

ICML 2026 · Optimization › Optimizers · approach: Zeroth-order training methods

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
sample independent directions across radii instead of sharing one direction (ablation)Romberg-ZOGE (R=2) · Poly8/Rosenbrock directional varianceleading variance coefficient: approximately 2.22Hurt
use Romberg extrapolation (R=2) instead of two-point, spherical directionsZO prompt tuning OPT-1.3B · SST-2test accuracy: 0.787 -> 0.912Improved
use Romberg extrapolation (R=2) instead of two-point, Gaussian directionsZO prompt tuning OPT-1.3B · SST-2test accuracy: 0.570 -> 0.873Improved
use Romberg-ZOGE instead of P4 unbiased estimatorZO prompt tuning OPT-1.3B · SST-2test accuracy: 0.877 -> 0.912Improved
use kernel-based estimator instead of two-pointZO prompt tuning OPT-1.3B · SST-2test accuracy: 0.787 -> 0.552Hurt
use spherical instead of Gaussian directions for Romberg-ZOGE (ablation)ZO prompt tuning OPT-1.3B · SST-2test accuracy: 0.873 -> 0.912Improved
use Romberg-ZOGE (R=2) instead of two-point in ZO-SGD with equal evaluations per iterationZO-SGD · Poly8 and Rosenbrock (n=100)objective valueImproved
use Romberg-ZOGE (R=2) instead of two-point under equal evaluations per iterationZO optimization · wireless network smooth-quantile SEobjective valueImproved
increase extrapolation order R under a stochastic oracle (ablation)Romberg-ZOGE · stochastic oracle theorybias and noise amplificationMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy