Research map / Optimization / Optimizers
Stability and Generalization of Bilevel Programming in Hyperparameter Optimization
Fan Bao, Guoqiang Wu, Chongxuan Li, Jun Zhu, Bo Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use large inner-loop K and outer-loop T in unrolled differentiation | Unrolled differentiation · Omniglot feature learning and MNIST data reweighting | testing loss | Hurt |
| Use small inner-loop K and outer-loop T in unrolled differentiation | Unrolled differentiation · Omniglot feature learning and MNIST data reweighting | validation fit | Hurt |
| Reduce the inner-loop learning rate and increase K | Unrolled differentiation · Feature learning and data reweighting | testing loss | Hurt |
| Use random-search cross-validation instead of unrolled differentiation in a high-dimensional hyperparameter space | Cross-validation versus unrolled differentiation · Omniglot feature learning | testing loss | Hurt |
| Use cross-validation instead of unrolled differentiation when the hyperparameter space is smaller | Cross-validation versus unrolled differentiation · Feature learning and data reweighting with fewer hyperparameters | testing loss | Improved |
| Add weight decay to the outer level (ablation) | Unrolled differentiation · Omniglot feature learning and MNIST data reweighting | testing loss | Improved |
| Add weight decay to the inner level (ablation) | Unrolled differentiation · Omniglot feature learning and MNIST data reweighting | testing loss | Improved |
| Add weight decay to both levels instead of using the better single-level regularizer (ablation) | Unrolled differentiation · Omniglot feature learning and MNIST data reweighting | testing loss | Hurt |
| Use very heavy inner-level weight decay (ablation) | Unrolled differentiation · MNIST data reweighting | optimization stability | Hurt |
Closest papers by meaning
- Lower Bounds of Uniform Stability in Gradient-Based Bilevel Algorithms for Hyperparameter Optimization (NeurIPS 2024 · similarity 0.91)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (ICLR 2025 · similarity 0.89)
- Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis (ICLR 2024 · similarity 0.89)
- Unlocking Global Optimality in Bilevel Optimization: A Pilot Study (ICLR 2025 · similarity 0.87)
- Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization (ICML 2025 · similarity 0.87)
- A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
