Research map / Architectures and efficiency / RNNs and normalization
Training Stronger Baselines for Learning to Optimize
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add progressive curriculum learning that increases unroll length during training (ablation) | L2O-DM · MLP-orig (MNIST) | training loss | Helped |
| Add off-policy imitation learning from Adam, SGD, and Adagrad (ablation) | L2O-DM · MLP-ReLU, Conv-MNIST, and Conv-CIFAR | training loss | Helped |
| Combine progressive curriculum learning with off-policy imitation learning (ablation) | L2O-DM · MLP and CNN optimizees | training loss | Helped |
| Increase training epochs without increasing optimizee trajectory length (ablation) | L2O-DM · MLP-orig (MNIST) | training loss | Hurt |
| Train with longer optimization trajectories as training epochs increase (ablation) | L2O-DM · MLP-orig (MNIST) | training loss | Helped |
| Add curriculum learning to L2O-DM (ablation) | L2O-DM · MLP-orig (MNIST) | training loss: less than 1/14 training iterations | Helped |
| Add imitation learning to L2O-DM (ablation) | L2O-DM · MLP-ReLU, Conv-MNIST, and Conv-CIFAR | training stability | Helped |
| Use curriculum learning rather than imitation learning (ablation) | L2O-DM · MLP optimizees | training loss | Helped |
| Use imitation learning rather than curriculum learning (ablation) | L2O-DM · CNN optimizees | training loss | Helped |
| Use off-policy imitation learning instead of self-improving mixed optimizer trajectories (ablation) | L2O-Scale · Optimizee test settings | training loss | Helped |
| Apply curriculum learning and imitation learning | RNNprop · Five optimizee test settings | training loss | Helped |
| Apply curriculum learning and imitation learning | L2O-Scale · Five optimizee test settings | training loss | Helped |
Closest papers by meaning
- M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation (ICLR 2023 · similarity 0.93)
- Towards Constituting Mathematical Structures for Learning to Optimize (ICML 2023 · similarity 0.91)
- Learning Provably Improves the Convergence of Gradient Descent (NeurIPS 2025 · similarity 0.90)
- Towards Robust Learning to Optimize with Theoretical Guarantees (CVPR 2024 · similarity 0.88)
- MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization (NeurIPS 2025 · similarity 0.86)
- Investigation into Training Dynamics of Learned Optimizers (Student Abstract) (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.