Research map / Optimization / Optimizers
Implicit differentiation of Lasso-type models for hyperparameter optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use implicit forward differentiation instead of forward iterative differentiation | Lasso hyperparameter optimization · Held-out loss on rcv1, 20news, and finance | time to optimize validation loss | Helped |
| Use implicit forward differentiation instead of implicit differentiation with a linear solve | Lasso hyperparameter optimization · Held-out loss on 20news | time to optimize validation loss | Helped |
| Use implicit forward differentiation rather than forward iterative differentiation | MCP hyperparameter optimization · Held-out criterion | optimization performance | Helped |
| Optimize the regularization parameter continuously with a gradient-based method rather than grid or Bayesian search | Lasso hyperparameter optimization · SURE, synthetic data | MSE | Helped |
| Use implicit forward differentiation rather than other hypergradient strategies | Lasso hyperparameter optimization · SURE, synthetic data | hyperparameter-search runtime | Helped |
| Use weighted Lasso instead of Lasso | Regularized linear regression · SURE, synthetic data | MSE | Helped |
| Use implicit forward differentiation rather than naive forward or backward iterative differentiation | Weighted Lasso hyperparameter optimization · SURE, synthetic data | hyperparameter-search runtime | Helped |
| Use implicit differentiation with a support-restricted linear solve | Weighted Lasso hyperparameter optimization · SURE, synthetic data | hyperparameter-search runtime | No clear effect |
Closest papers by meaning
- Scalable One-Pass Optimisation of High-Dimensional Weight-Update Hyperparameters by Implicit Differentiation (ICLR 2022 · similarity 0.87)
- Implicit Sparse Regularization: The Impact of Depth and Early Stopping (NeurIPS 2021 · similarity 0.86)
- Implicit Regularization for Optimal Sparse Recovery (NeurIPS 2019 · similarity 0.86)
- One-step differentiation of iterative algorithms (NeurIPS 2023 · similarity 0.86)
- Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators (NeurIPS 2024 · similarity 0.84)
- Mask in the Mirror: Implicit Sparsification (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.