Research map / Optimization / Nonconvex and smooth optimization
Provably Faster Algorithms for Bilevel Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use momentum-based recursive mini-batch estimators for both the inner gradient and outer hypergradient | Stochastic bilevel optimization · Nonconvex-strongly-convex bilevel optimization | gradient complexity: \mathcal{O}(\epsilon^{-2}) -> \mathcal{O}(\epsilon^{-1.5}) | Helped |
| Use recursive variance reduction with periodic large-batch estimates in nested loops | Stochastic bilevel optimization · Nonconvex-strongly-convex bilevel optimization | gradient complexity: \mathcal{O}(\epsilon^{-2}) -> \mathcal{\widetilde O}(\epsilon^{-1.5}) | Helped |
| Use a double-loop design rather than a single-loop design | Bilevel optimizers · MNIST hyper-cleaning | training loss convergence versus running time | Helped |
| Increase VRBO inner-loop steps from 1 to 20 (ablation) | VRBO · MNIST hyper-cleaning | training loss and convergence stability | Helped |
| Increase VRBO inner-loop steps beyond 20 (ablation) | VRBO · MNIST hyper-cleaning | training loss and convergence stability | Hurt |
| Use double-loop rather than single-loop updates | Bilevel optimizers · 20 Newsgroup logistic regression | test accuracy | Mixed |
Closest papers by meaning
- Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling (NeurIPS 2024 · similarity 0.90)
- Double Momentum Method for Lower-Level Constrained Bilevel Optimization (ICML 2024 · similarity 0.90)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (ICLR 2025 · similarity 0.89)
- An Accelerated Gradient Method for Convex Smooth Simple Bilevel Optimization (NeurIPS 2024 · similarity 0.89)
- A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness (ICML 2024 · similarity 0.89)
- Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.