Research map / Optimization / Optimizers
Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use STN to tune regularization hyperparameters online | 2-layer LSTM · PTB validation | perplexity: 72.13 -> 70.30 | Helped |
| Use STN to tune regularization hyperparameters online | 2-layer LSTM · PTB test | perplexity: 69.29 -> 67.68 | Helped |
| Use STN to tune regularization and augmentation hyperparameters | AlexNet · CIFAR-10 validation | loss: 0.636 -> 0.575 | Helped |
| Use STN to tune regularization and augmentation hyperparameters | AlexNet · CIFAR-10 test | loss: 0.651 -> 0.576 | Helped |
| Replace fixed output dropout p=0.68 with the STN-learned schedule | LSTM · PTB validation | perplexity: 85.83 -> 82.58 | Helped |
| Replace fixed output dropout p=0.68 with the STN-learned schedule | LSTM · PTB test | perplexity: 83.19 -> 79.02 | Helped |
| Use the STN-learned output-dropout schedule instead of fixed p=0.68 | standard LSTM · PTB validation | perplexity: 85.83 -> 82.87 | Helped |
| Use the STN-learned output-dropout schedule instead of fixed p=0.68 | standard LSTM · PTB test | perplexity: 83.19 -> 79.93 | Helped |
| Use fixed output dropout p=0.78 instead of following the STN schedule | LSTM · PTB validation | perplexity: 82.58 -> 89.65 | Hurt |
| Use fixed output dropout p=0.78 instead of following the STN schedule | LSTM · PTB test | perplexity: 79.02 -> 86.90 | Hurt |
Closest papers by meaning
- Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function (NeurIPS 2025 · similarity 0.88)
- On the Parameterization of Second-Order Optimization Effective towards the Infinite Width (ICLR 2024 · similarity 0.88)
- Principled Architecture-aware Scaling of Hyperparameters (ICLR 2024 · similarity 0.87)
- Accelerating neural network training: An analysis of the AlgoPerf competition (ICLR 2025 · similarity 0.87)
- Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis (ICLR 2024 · similarity 0.86)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.