Research map / Optimization / Implicit bias and generalization
Deep Bilevel Learning
Simon Jenni, Paolo Favaro
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add L1 normalization of the mini-batch weights (ablation) | Bilevel (CifarNet) · CIFAR-10 clean labels | clean test accuracy: 74.32 -> 75.52 | Improved |
| add L1 normalization of the mini-batch weights (ablation) | Bilevel (Inception) · CIFAR-10 clean labels | clean test accuracy: 77.07 -> 87.78 | Improved |
| force identical label distribution across compared mini-batches (ablation) | Bilevel (Inception) · CIFAR-10 50% random label noise | clean test accuracy: 27.20 -> 73.08 | Improved |
| force identical label distribution across compared mini-batches (ablation) | Bilevel (CifarNet) · CIFAR-10 clean labels | clean test accuracy: 68.19 -> 75.52 | Improved |
| compute separate weights per layer instead of one per mini-batch (ablation) | Bilevel (CifarNet) · CIFAR-10 50% random label noise | clean test accuracy: 47.62 -> 49.62 | Improved |
| compute separate weights per layer instead of one per mini-batch (ablation) | Bilevel (Inception) · CIFAR-10 clean labels | clean test accuracy: 87.78 -> 85.25 | Hurt |
| allow unequal dropout across compared mini-batches (ablation) | Bilevel (Inception) · CIFAR-10 50% random label noise | clean test accuracy: 73.08 -> 75.71 | Improved |
| increase number of compared mini-batches (ablation) | Bilevel · CIFAR-10 (CifarNet and Inception) | clean test accuracy | Mixed |
| increase validation ratio (ablation) | Bilevel · CIFAR-10 | test accuracy | No clear effect |
| vary the hyper-parameter mu (ablation) | Bilevel · CIFAR-10 | test accuracy | No clear effect |
| use bilevel optimization instead of standard SGD | PreAct ResNet-18 · CIFAR-10 (40% label noise) | test accuracy: 69.6% -> 87.0% | Improved |
| use bilevel optimization instead of standard SGD | PreAct ResNet-18 · CIFAR-100 (40% label noise) | test accuracy: 44.9% -> 59.8% | Improved |
Closest papers by meaning
- Stability and Generalization of Bilevel Programming in Hyperparameter Optimization (NeurIPS 2021 · similarity 0.90)
- On Generalization in Bilevel Optimization with Overparameterized Models (NeurIPS 2026 · similarity 0.90)
- Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions (ICLR 2025 · similarity 0.87)
- Cross-regularization: Adaptive Model Complexity through Validation Gradients (ICML 2025 · similarity 0.87)
- Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis (ICLR 2024 · similarity 0.86)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
