Research map / Optimization / Optimizers
Parabolic Approximation Line Search for DNNs
Maximus Mutschler, Andreas Zell
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace PAL with an exact binary line search measuring up to 20 values per line (ablation) | PAL · ResNet32 CIFAR-10 | training performance | Hurt |
| add dynamic step sizes when no direction adaptation is used (ablation) | PAL · EfficientNet CIFAR-10 | performance | Improved |
| add dynamic step sizes when direction adaptation is used (ablation) | PAL · EfficientNet CIFAR-10 | performance | Mixed |
| add direction adaptation factor beta (0.2 or 0.4) (ablation) | PAL · EfficientNet CIFAR-10 | performance | Mixed |
| use constant beta 0.2 or 0.4 instead of Fletcher-Reeves, Polak-Ribiere, Hestenes-Stiefel, Dai-Yuan formulas (ablation) | PAL · CIFAR-10 | performance | No clear effect |
| increase update step adaptation alpha to 1.4 (ablation) | PAL · ResNet32 CIFAR-10 | performance | Improved |
| vary maximum step size s_max (ablation) | PAL · ResNet32 CIFAR-10 | performance | No clear effect |
| beta above 0.6 (ablation) | PAL · ResNet32 CIFAR-10 | performance | Hurt |
| take measuring step size mu outside 10^-2 to 10^-0.5 (ablation) | PAL · ResNet32 CIFAR-10 | performance | Hurt |
| compare PAL against basic SGD/ADAM/RMSProp without step size schedule | PAL · CIFAR-10/100, ImageNet | validation/test accuracy | Improved |
| compare PAL against tuned-schedule SGD | PAL · CIFAR-10/100, ImageNet | test accuracy | Hurt |
| compare PAL against SLS | PAL · CIFAR-10/100, ImageNet | validation accuracy | Improved |
Closest papers by meaning
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.87)
- Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size (NeurIPS 2025 · similarity 0.86)
- On the Parameterization of Second-Order Optimization Effective towards the Infinite Width (ICLR 2024 · similarity 0.85)
- Loss Landscape Characterization of Neural Networks without Over-Parametrization (NeurIPS 2024 · similarity 0.85)
- Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning (NeurIPS 2026 · similarity 0.85)
- Power Lines: Scaling laws for weight decay and batch size in LLM pre-training (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
