Research map / Optimization / Optimizers
Stochastic Chebyshev Gradient Descent for Spectral Optimization
Insu Han, Haim Avron, Jinwoo Shin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use unbiased randomized Chebyshev gradients with the optimal degree distribution | SGD for matrix completion · MovieLens 1M | convergence: about 2 times faster | Improved |
| use unbiased randomized Chebyshev gradients with the optimal degree distribution | SVRG for matrix completion · MovieLens 10M | speedup to RMSE 1.8: up to 6 times faster | Improved |
| use unbiased randomized Chebyshev gradients | SGD for Gaussian process regression · Natural sound dataset | speedup to RMSE 0.0375: 2 times faster | Improved |
| use unbiased randomized Chebyshev gradients instead of biased gradient estimates | SGD for Gaussian process regression · Szeged humid dataset | convergence | Improved |
| replace randomized degree sampling with a deterministic polynomial degree (ablation) | SGD for matrix completion · MovieLens datasets | convergence | Hurt |
| replace the optimal degree distribution with a negative-binomial distribution | SGD · MovieLens 1M | convergence | Hurt |
| replace the optimal degree distribution with a Poisson distribution | SGD · MovieLens 1M | convergence | Hurt |
| increase the deterministic polynomial degree to N=30 (ablation) | SGD-DET · MovieLens 1M | bias error | Improved |
| vary expected randomized polynomial degree between N=10 and N=30 (ablation) | SGD · MovieLens 1M | error | No clear effect |
| use SVRG instead of GD | matrix completion · MovieLens 1M and MovieLens 10M | convergence speed: about 2 times faster; up to 6 times faster | Improved |
| increase the dataset dimension and parameter count | SGD for matrix completion · MovieLens 10M versus MovieLens 1M | convergence | Hurt |
Closest papers by meaning
- Stochastic Bias-Reduced Gradient Methods (NeurIPS 2021 · similarity 0.87)
- Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithms (NeurIPS 2024 · similarity 0.86)
- SOREL: A Stochastic Algorithm for Spectral Risks Minimization (ICLR 2025 · similarity 0.86)
- First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities (NeurIPS 2023 · similarity 0.86)
- Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives (NeurIPS 2026 · similarity 0.85)
- Spectral Preconditioning for Gradient Methods on Graded Non-convex Functions (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
