Research map / Optimization / Nonconvex and smooth optimization
Global Optimization with a Power-Transformed Objective and Gaussian Smoothing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GS-PowerOpt (GSPTO): raise the objective to a large power (or exponential power) before Gaussian smoothing and run normalized stochastic gradient ascent, with theory that the smoothed optimum approaches the global optimum and experiments on benchmark functions and black-box adversarial attacks.
Problem
Global optimization of possibly non-differentiable, non-concave objectives; homotopy and single-loop Gaussian smoothing methods only reliably reach local optima or are slow.
Approach
Apply a (exponential) power transformation to f so the global maximum gets much more weight, then Gaussian-smooth and optimize with zeroth-order stochastic gradient ascent; the smoothed maximizer converges to a neighborhood of the global optimum as the power N grows.
Key results
- On MNIST targeted attack, EPGS reaches 100% success with average R^2 of 0.85 versus 0.54 for the next smoothing baseline ZO-SLGHd.
- On CIFAR-10 targeted attack, EPGS reaches 100% success with R^2 0.98, close to ZO-SLGHd's 0.99.
- On Rosenbrock, EPGS finds (0.994, 1.002) with f=-0.18, while STD-Homotopy finds f=-2.401 and ZO-SGD f=-121.14.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EPGS instead of ZO-SLGHr | Ackley optimization · Ackley | fitness: 22.621 -> 22.682 | Helped |
| Use EPGS instead of STD-Homotopy | Rosenbrock optimization · Rosenbrock | fitness: -2.401 -> -0.18 | Helped |
| Use EPGS instead of PSO | Rosenbrock optimization · Rosenbrock | fitness: 0.0 -> -0.18 | Hurt |
| Use EPGS instead of ZO-SLGHd | MNIST targeted attack · MNIST | R²: .54\,(.14) -> .85\, (.05) | Helped |
| Use EPGS instead of ZO-AdaMM | MNIST targeted attack · MNIST | iterations to optimal perturbation: 50\,(25) -> 438\,(120) | Hurt |
| Use EPGS instead of ZO-SLGHd | CIFAR-10 targeted attack · CIFAR-10 | R²: .99\,(.01) -> .98\, (.02) | Hurt |
| Increase the power N in PGS or EPGS | PGS/EPGS on the toy objective · Toy objective, d=2 and d=5 | MSE between solution and global optimum | Helped |
Surprising findings
- ZO-SGD on MNIST attack has an average R^2 of -3.30, indicating perturbations far larger than the image.
Limitations
- Convergence guarantee is to a local maximum of the smoothed objective within a delta-neighborhood of the global optimum, requiring large N.
- Large N causes numerical overflow, requiring gradient reformulation.
- Experiments are small-scale (2D/5D benchmarks, one classifier per dataset) with hyperparameters tuned by trials on the test setting.
Experimental setup
Datasets: MNIST CIFAR-10
Benchmarks: Ackley Rosenbrock MNIST targeted adversarial attack CIFAR-10 targeted adversarial attack
Metrics: best fitness value f(mu*) success rate average R^2 iterations taken
Baselines: STD-Homotopy ZO-SLGHd ZO-SLGHr ZO-SGD ZO-AdaMM PSO
Builds on: Gaussian homotopy single-loop Gaussian homotopy ZO-SGD ZO-AdaMM
Closest papers by meaning
- Stochastic Optimization with Random Search (NeurIPS 2026 · similarity 0.85)
- Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization (ICML 2024 · similarity 0.85)
- Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity (NeurIPS 2025 · similarity 0.85)
- Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster (ICML 2025 · similarity 0.84)
- Skip the Hessian, Keep the Rates: Globalized Semismooth Newton with Lazy Hessian Updates (NeurIPS 2026 · similarity 0.84)
- Non-asymptotic Global Convergence Analysis of BFGS with the Armijo-Wolfe Line Search (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.