Research map / Optimization / Nonconvex and smooth optimization
Stochastic DCA with Variance Reduction and Applications in Machine Learning
Hoai An Le Thi, Hoang Phuc Hau Luu, Hoai Minh Le, Tao Pham Dinh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sampling without replacement instead of with replacement in DCA-SAGA (ablation) | DCA-SAGA · Nonnegative PCA, 10 LIBSVM datasets | suboptimality: DCA-SAGA-v2 usually below 10^-10; DCA-SAGA-v1 much worse | Improved |
| use sampling without replacement in DCA-SAGA (ablation) | DCA-SAGA · Group variable selection in multiclass logistic regression | objective value | Improved |
| use sampling without replacement in DCA-SAGA (ablation) | DCA-SAGA · Sparse linear regression | objective value | Improved |
| use SVRG variant v2 instead of v1 in DCA-SVRG (ablation) | DCA-SVRG · Nonnegative PCA, 10 LIBSVM datasets | suboptimality | No clear effect |
| replace SDCA with DCA-SAGA | SDCA · Group variable selection in multiclass logistic regression | objective gain: 0.71% up to 60.15% | Improved |
| replace SDCA with DCA-SVRG-v1 | SDCA · Group variable selection in multiclass logistic regression | objective gain: 0.56% to 54.73% | Improved |
| replace DCA-SVRG-v1 with DCA-SAGA | DCA-SVRG-v1 · Sparse linear regression | objective gain: 5.22%, 0.24%, 0.56% on cadata, sgemm product, YearPredictionMSD | Improved |
| replace SDCA with DCA-SVRG-v1 | SDCA · Sparse linear regression | objective gain: 2.11%, 11.45%, 0.98% | Improved |
| use proximal operator directly (MB-SPG) instead of Moreau envelope (SSDC) | SSDC · Sparse linear regression | objective value | Improved |
| use DCA-SVRG instead of prox-SAGA/SDCA/prox-SGD | stochastic baselines · Nonnegative PCA, 10 LIBSVM datasets | suboptimality: around 10^-15 vs SDCA slightly worse than prox-SAGA (<10^-5), prox-SGD stuck around 10^-5 | Improved |
Closest papers by meaning
- Stochastic DCA for the Large-sum of Non-convex Functions Problem and its Application to Group Variable Selection in Classification (ICML 2017 · similarity 0.92)
- Stochastic Variance Reduction Methods for Saddle-Point Problems (NeurIPS 2016 · similarity 0.90)
- A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates (ICML 2018 · similarity 0.89)
- Stochastic PCA with $\ell_2$ and $\ell_1$ Regularization (ICML 2018 · similarity 0.88)
- Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite Sum Structure (NeurIPS 2017 · similarity 0.88)
- Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence (ICML 2019 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
