Research map / Optimization / Nonconvex and smooth optimization
Lower-Level Agnostic Bilevel Optimization
Peiwen Qiu, Prashant Khanduri, Jia (Kevin) Liu
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a bilevel optimization method for settings where the lower-level problem is a black box (solver, simulator, adversary), so the lower-level objective cannot be evaluated, differentiated, or unrolled.
Problem
Bilevel optimization where the lower-level response comes from a black-box solver, simulator, or adversary, so standard methods that assume access to the lower-level objective (for evaluation, differentiation, or unrolling) fail.
Approach
Lower-level-agnostic bilevel optimization: the upper-level procedure only uses the output of the lower-level response, without requiring access to the lower-level objective, its gradients, or unrolled iterations. Details are not available beyond the abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LeGo-BiO (LL-agnostic) instead of FAST-AT / FAST-AT-GA | adversarial training PreActResNet-18 · CIFAR-10 | standard and robust accuracy: standard 73.34 -> 77.18; robust 32.56 -> 39.08 (vs FAST-AT) | Improved |
| use LeGo-BiO (no LL objective access) instead of FAST-BAT | adversarial training · CIFAR-10 | standard / robust accuracy: standard 76.04 -> 77.18; robust 39.84 -> 39.08 | Mixed |
| use LeGo-BiO instead of zeroth-order ZDSBA | deep hyper-representation on MNIST LeNet · MNIST | training loss | Improved |
| reduce LL approximation error of the response oracle (ablation) | LeGo-BiO · MNIST deep hyper-representation | iterations to reach same training loss | Improved |
| vary LL approximation error level (ablation) | LeGo-BiO · MNIST deep hyper-representation | final training loss | No clear effect |
| swap LL solver among SGD, Adam and SciPy (ablation) | LeGo-BiO · MNIST deep hyper-representation | training loss | Mixed |
| use LeGo-BiO instead of SOBA/MA-SOBA | data weighting for Llama-3.2-1B-Instruct on HH-RLHF · HH-RLHF | convergence speed | Improved |
Closest papers by meaning
- First-Order Minimax Bilevel Optimization (NeurIPS 2024 · similarity 0.88)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (ICLR 2025 · similarity 0.88)
- qNBO: quasi-Newton Meets Bilevel Optimization (ICLR 2025 · similarity 0.88)
- Conditional Gradient Methods with Standard LMO for Stochastic Simple Bilevel Optimization (NeurIPS 2025 · similarity 0.87)
- Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions (NeurIPS 2025 · similarity 0.87)
- Unlocking Global Optimality in Bilevel Optimization: A Pilot Study (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
