Research map / Optimization / Nonconvex and smooth optimization
Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters
Zeyuan Allen-Zhu, Yang Yuan, Karthik Sridharan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use raw clustering in SVRG-style estimator (ClusterSVRG) vs SVRG/SAGA at large lambda | ClusterSVRG · Covtype/SensIT ridge/Lasso | convergence vs #grad/n | Improved |
| use clustering in accelerated method (ClusterACDM) vs ACDM and others at small lambda | ClusterACDM · Covtype/SensIT | convergence vs #grad/n | Improved |
| apply cluster methods on poorly clusterable data | ClusterSVRG and ClusterACDM · News20 | convergence vs #grad/n | No clear effect |
| use clustering with smaller diameter delta (ablation) | ClusterSVRG · Covtype/SensIT | convergence | Improved |
| use clustering with larger diameter delta (ablation) | ClusterACDM · Covtype/SensIT | convergence | Improved |
| vary clustering quality across 7 clusterings (ablation) | ClusterSVRG and ClusterACDM · SensIT, Covtype | convergence | No clear effect |
| add clustering preprocessing cost | ClusterSVRG · Covtype/SensIT | preprocessing time: 3 passes of SAGA | No clear effect |
Closest papers by meaning
- On the Fine-Grained Complexity of Empirical Risk Minimization: Kernel Methods and Neural Networks (NeurIPS 2017 · similarity 0.87)
- Scaling-up Empirical Risk Minimization: Optimization of Incomplete $U$-statistics (JMLR 2016 · similarity 0.87)
- Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method for Regularized Empirical Risk Minimization (NeurIPS 2017 · similarity 0.86)
- Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity (NeurIPS 2018 · similarity 0.86)
- First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization (NeurIPS 2017 · similarity 0.86)
- On the Efficiency of ERM in Feature Learning (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
