AI Paper Atlas

Research map / Optimization / Nonconvex and smooth optimization

Implicit Differentiation for Fast Hyperparameter Selection in Non-Smooth Convex Learning

Quentin Bertrand, Quentin Klopfenstein, Mathurin Massias, Mathieu Blondel, Samuel Vaiter, Alexandre Gramfort, Joseph Salmon

JMLR 2022 · Optimization › Nonconvex and smooth optimization · approach: Coordinate and Adam convergence analysis · 17 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use forward differentiation of proximal coordinate descent instead of cvxpylayers for the Lasso hypergradientLasso hypergradient · gina (random columns)computation time: one order of magnitude faster than cvxpylayersImproved
use forward differentiation instead of backward differentiation of proximal coordinate descentLasso hypergradient · gina (random columns)computation time: two orders of magnitude faster than the backward differentiationImproved
restrict gradient computation to the support of the Jacobian (implicit differentiation) instead of forward differentiationLasso and SVM hypergradient · three data sets, two regularization valuestime to reach exact hypergradientImproved
solve the inner problem with Celer instead of vanilla proximal coordinate descentimplicit differentiation for Lasso · hold-out criteriontime to reach exact hypergradientImproved
use first-order hypergradient method instead of grid search and SMBOLasso CV · Lasso cross-validationtime to find optimal lambda: requiring only 5 iterationsImproved
use first-order methods instead of SMBOelastic net CV · rcv1, real-sim, news20time to convergeImproved
use first-order methods instead of zero-order solvers when classes are fewmulticlass sparse logistic regression · mnist, usps (q=10)multiclass cross-entropyNo clear effect
use first-order methods instead of zero-order solvers when classes are manymulticlass sparse logistic regression · rcv1 (q=53), aloi (q=1000)accuracyImproved
use one hyperparameter per class instead of a single shared regularization hyperparameter (ablation)multiclass logistic regression · multiclass datasetstest accuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy