Research map / Optimization / Optimizers
Bilevel Programming for Hyperparameter Optimization and Meta-Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn a shared representation with RN and train task-specific classifiers using bilevel training/validation splits | MiniImagenet few-shot learning · MiniImagenet 5 classes, 1-shot | accuracy: 50.54 ± 0.85 vs 55.71±0.99 | Mixed |
| use a residual network rather than C4L for the representation (ablation) | Hyper-representation · MiniImagenet few-shot learning | accuracy: relative improvement of 6.5% on one-shot and 4.2% on five-shot | Helped |
| use a bilevel representation objective with training/validation splitting rather than training-set loss (ablation) | Hyper-representation-C4L · MiniImagenet 5 classes, 1-shot | accuracy: 29.63 -> 47.51 | Helped |
| include the inner optimization dynamics when computing the hypergradient (ablation) | Hyper-representation-C4L · MiniImagenet 5 classes, 1-shot | accuracy: 41.12 -> 47.51 | Helped |
| use validation sets and the full hypergradient instead of training sets with an approximate hypergradient (ablation) | Hyper-representation-C4L · MiniImagenet 5 classes, 1-shot | accuracy: 38.80 -> 47.51 | Helped |
| learn the shared representation bilevelly rather than jointly optimizing it with task classifiers on training losses (ablation) | Hyper-representation-C4L · MiniImagenet 5 classes, 1-shot | accuracy: 40.46 -> 47.51 | Helped |
| increase inner optimization steps from 10 to 250 (ablation) | Bilevel ridge regression hyperparameter optimization · Synthetic regression test set | MAPE: 43.49 -> 0.50 | Helped |
| use 250 inner iterations instead of exact inner optimization (ablation) | Bilevel ridge regression hyperparameter optimization · Synthetic regression test set | MAPE: 0.50 vs 0.57 | No clear effect |
| use a small number of inner optimization steps as implicit regularization (ablation) | Linear hyper-representation optimization · Omniglot-derived hyperparameter optimization problem | test accuracy | Helped |
Closest papers by meaning
- Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis (ICLR 2024 · similarity 0.90)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (ICLR 2025 · similarity 0.88)
- qNBO: quasi-Newton Meets Bilevel Optimization (ICLR 2025 · similarity 0.88)
- Unlocking Global Optimality in Bilevel Optimization: A Pilot Study (ICLR 2025 · similarity 0.87)
- Scalable Meta-Learning via Mixed-Mode Differentiation (ICML 2025 · similarity 0.87)
- Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.