Research map / Learning theory and algorithms / Matrix and tensor methods
Orthant-Wise Passive Descent Algorithms for Training L1-Regularized Models
Jianqiao Wangni
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OPDA instead of a proximal algorithm | OPDA · Figure 1 synthetic L1 problem | final objective P(x): 34.621 -> 20.000 | Improved |
| use OPDA-FM instead of a proximal algorithm | OPDA-FM · Figure 2 nonconvex synthetic problem | final objective P(x): 2468.755 -> 177.038 | Improved |
| use OPDA-FM instead of Proximal-SVRG | OPDA-FM · rcv1 with L2 regularization lambda2 from 0 to 100/N | suboptimality | Improved |
| decrease the strong convexity coefficient mu | OPDA-FM vs Proximal-SVRG · rcv1 | suboptimality gap | Improved |
| increase the L1 regularization strength (4, 8, 32 times base) | OPDA-FM vs Proximal-SVRG · rcv1 | suboptimality gap | Improved |
| use OPDA-FM with no L2 regularization (non-strongly convex) | OPDA-FM · rcv1 lambda2 = 0 | suboptimality | Improved |
| use OPDA-QN instead of Proximal-SVRG | OPDA-QN · rcv1 and covtype | suboptimality vs datapasses and time | Improved |
| use Gaussian or previous-direction sketching instead of identity (BFGS) | OPDA-QN · covtype (low-dimensional) | suboptimality | Improved |
| use identity sketching (BFGS) instead of previous directions | OPDA-QN · rcv1 (high-dimensional) | datapasses and running time | Improved |
| use Gaussian sketching instead of BFGS or prev | OPDA-QN · rcv1 (high-dimensional) | suboptimality | Hurt |
| use OPDA-QN instead of OPDA-FM | OPDA · rcv1 and covtype | convergence | Improved |
| use OPDA-SGD instead of Proximal-SGD | OPDA-SGD · CIFAR10 sparse CNN | objective vs datapasses | Improved |
Closest papers by meaning
- L0Learn: A Scalable Package for Sparse Learning using L0 Regularization (JMLR 2023 · similarity 0.85)
- Adaptive Proximal Gradient Methods for Structured Neural Networks (NeurIPS 2021 · similarity 0.85)
- The Benefits of Implicit Regularization from SGD in Least Squares Problems (NeurIPS 2021 · similarity 0.85)
- Regularized Adaptive Momentum Dual Averaging with an Efficient Inexact Subproblem Solver for Training Structured Neural Network (NeurIPS 2024 · similarity 0.84)
- Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization (NeurIPS 2025 · similarity 0.84)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
