Research map / Optimization / Nonconvex and smooth optimization
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVRG variance reduction with a two-point random gradient estimator instead of ZO-SGD | ZO-SVRG · Nonconvex stochastic optimization | convergence rate (expected squared gradient norm): O(1/√T) -> O(1/T), with an additional O(1/b) term | Helped |
| Average multiple random direction samples (ablation) | ZO-SVRG · Nonconvex stochastic optimization | convergence rate (expected squared gradient norm): O(d/T + 1/b) -> O(d/T + 1/(b min{d,q})) | Helped |
| Replace the random-direction estimator with the coordinate-wise gradient estimator (ablation) | ZO-SVRG · Nonconvex stochastic optimization | convergence rate (expected squared gradient norm): O(d/T + 1/b) -> O(d/T) | Helped |
| Use coordinate-wise gradient estimation, which requires more function queries (ablation) | ZO-SVRG-Coord · Chemical material classification test | testing error (%): 11.18% -> 20.67% | Hurt |
| Increase the mini-batch size (ablation) | ZO-SVRG · Chemical material classification training | training loss | Helped |
| Use a small mini-batch size (b ≤ 10) (ablation) | ZO-SVRG · Chemical material classification training | convergence to a critical point: O(1/b) | Hurt |
| Increase the number of random direction samples q (ablation) | ZO-SVRG-Ave · Black-box MNIST universal adversarial perturbations | convergence trajectory stability: 30× (q=10), 77× (q=20), and 380× (q=30) more function evaluations than ZO-SGD to reach attack loss 7 | Helped |
| Use ZO-SVRG-Ave with q=30 instead of ZO-SGD | universal adversarial perturbation generation · Black-box MNIST universal adversarial perturbations | relative ℓ2 distortion improvement: 30% | Helped |
| Use ZO-SVRG-Ave instead of ZO-SGD | universal adversarial perturbation generation · Black-box MNIST universal adversarial perturbations | function evaluations to reach attack loss 7: roughly 30× (q=10), 77× (q=20) and 380× (q=30) more function evaluations | Hurt |
| Use ZO-SVRG-Ave (q=1), equivalent to ZO-SVRG, rather than the averaged estimator (ablation) | ZO-SVRG-Ave · Nonconvex stochastic optimization | variance-related convergence correction: O(1/(b min{d,q})) | Hurt |
| Increase q beyond the dimension d (ablation) | ZO-SVRG-Ave · Nonconvex stochastic optimization | convergence rate: q ≥ d | No clear effect |
| Use ZO-SVRG instead of ZO-SVRC | chemical material classification · Chemical material classification test | testing error (%): 23.70% -> 11.18% | Helped |
Closest papers by meaning
- Refining Adaptive Zeroth-Order Optimization at Ease (ICML 2025 · similarity 0.91)
- Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient (ICML 2024 · similarity 0.90)
- A Coefficient Makes SVRG Effective (ICLR 2025 · similarity 0.89)
- Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise (NeurIPS 2026 · similarity 0.88)
- Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization (NeurIPS 2026 · similarity 0.87)
- DeepZero: Scaling Up Zeroth-Order Optimization for Deep Model Training (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.