Research map / Optimization / Optimizers
Gradient Estimation with Simultaneous Perturbation and Compressive Sensing
Vivek S. Borkar, Vikranth R. Dwaracherla, Neeraja Sahasrabudhe
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use adaptive (few-iteration homotopy) l1-minimization inside gradient descent instead of full l1 solve each step | SP + compressive sensing gradient descent · sparse-gradient test function, n=25000 | time taken (s): 489 -> 39 | Improved |
| use adaptive l1-minimization instead of non-adaptive | SP + compressive sensing gradient descent · sparse-gradient test function, n=1000 | time taken (s): 50.572 -> 5.910 | Improved |
| use adaptive l1-minimization instead of non-adaptive | SP + compressive sensing gradient descent · sparse-gradient test function, n=10000 | time taken (s): 213.6 -> 30.4 | Improved |
| replace exact gradient with compressive-sensing estimated gradient (non-adaptive) | gradient descent · sparse-gradient test function, n=10000 | time taken (s): 5873.8 -> 213.6 | Improved |
| replace exact gradient with compressive-sensing estimated gradient (non-adaptive) | gradient descent · sparse-gradient test function, n=1000 | time taken (s): 171.150 -> 50.572 | Improved |
| use the proposed estimator instead of Sparse Gradient Learning for gradient descent | gradient descent with estimated gradient · n=10000, s=100, m=500 | time taken (s): 993 -> 83 | Improved |
| add compressive sensing recovery on top of naive SP gradient estimate (ablation) | SP gradient estimation · f(x)=x(1)^2+...+x(s)^2, sparse gradient | percentage gradient error | Improved |
| increase number of SP averaging iterations k (ablation) | SP + compressive sensing gradient estimator · n=25000, m=50 sparse quadratic | percentage gradient error | Improved |
| increase sparsity s with fixed measurements m=50 (ablation) | SP + compressive sensing gradient estimator · n=10000, m=50 | percentage gradient error | Hurt |
| increase sparsity s (ablation) | naive SP gradient estimate · sparse gradient test function | percentage gradient error | Improved |
Closest papers by meaning
- Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization (ICML 2024 · similarity 0.87)
- Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise (JMLR 2020 · similarity 0.86)
- Stochastic Bias-Reduced Gradient Methods (NeurIPS 2021 · similarity 0.86)
- Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations (ICLR 2025 · similarity 0.86)
- Implicit Regularization in Nonconvex Statistical Estimation: Gradient Descent Converges Linearly for Phase Retrieval and Matrix Completion (ICML 2018 · similarity 0.86)
- Implicit Regularization for Optimal Sparse Recovery (NeurIPS 2019 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
