Research map / Optimization / Nonconvex and smooth optimization
Implicit Differentiation for Fast Hyperparameter Selection in Non-Smooth Convex Learning
Quentin Bertrand, Quentin Klopfenstein, Mathurin Massias, Mathieu Blondel, Samuel Vaiter, Alexandre Gramfort, Joseph Salmon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use forward differentiation of proximal coordinate descent instead of cvxpylayers for the Lasso hypergradient | Lasso hypergradient · gina (random columns) | computation time: one order of magnitude faster than cvxpylayers | Improved |
| use forward differentiation instead of backward differentiation of proximal coordinate descent | Lasso hypergradient · gina (random columns) | computation time: two orders of magnitude faster than the backward differentiation | Improved |
| restrict gradient computation to the support of the Jacobian (implicit differentiation) instead of forward differentiation | Lasso and SVM hypergradient · three data sets, two regularization values | time to reach exact hypergradient | Improved |
| solve the inner problem with Celer instead of vanilla proximal coordinate descent | implicit differentiation for Lasso · hold-out criterion | time to reach exact hypergradient | Improved |
| use first-order hypergradient method instead of grid search and SMBO | Lasso CV · Lasso cross-validation | time to find optimal lambda: requiring only 5 iterations | Improved |
| use first-order methods instead of SMBO | elastic net CV · rcv1, real-sim, news20 | time to converge | Improved |
| use first-order methods instead of zero-order solvers when classes are few | multiclass sparse logistic regression · mnist, usps (q=10) | multiclass cross-entropy | No clear effect |
| use first-order methods instead of zero-order solvers when classes are many | multiclass sparse logistic regression · rcv1 (q=53), aloi (q=1000) | accuracy | Improved |
| use one hyperparameter per class instead of a single shared regularization hyperparameter (ablation) | multiclass logistic regression · multiclass datasets | test accuracy | No clear effect |
Closest papers by meaning
- Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity (NeurIPS 2025 · similarity 0.87)
- Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity (ICLR 2025 · similarity 0.87)
- Directional Smoothness and Gradient Methods: Convergence and Adaptivity (NeurIPS 2024 · similarity 0.87)
- Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates (ICML 2024 · similarity 0.86)
- A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning (AAAI 2025 · similarity 0.86)
- Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
