Research map / Optimization / Optimizers
Meta-Descent for Online, Continual Prediction
Andrew Jacobsen, Matthew Schlegel, Cameron Linke, Thomas Degris, Adam White, Martha White
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine AdaGain with RMSProp instead of using AdaGain with vanilla SGD (ablation) | AdaGain · Rosenbrock function | optimization performance | Improved |
| Use the linear-complexity approximation instead of full quadratic AdaGain (ablation) | AdaGain with RMSProp · Rosenbrock function | optimization performance | No clear effect |
| Estimate the Jacobian with finite differences instead of analytic derivatives (ablation) | Linear-complexity AdaGain · Rosenbrock function | optimization performance | No clear effect |
| Remove RMSProp from AdaGain (ablation) | AdaGain · Rosenbrock function | optimization performance | Hurt |
| Use AdaGain rather than AMSGrad for online prediction | TD(λ) on mobile robot data · Nexting robot sensor predictions | SMAPE | Improved |
| Use AdaGain rather than Adam for off-policy TD(λ) | Baird's counterexample · Baird's counterexample | value error | Improved |
| Adapt per-feature step-sizes with AdaGain rather than using a global step-size | TD(λ) · Baird's counterexample | divergence | Improved |
| Use AdaGain rather than AMSGrad | Online continual prediction · Stateless tracking problem | MSE | Mixed |
| Use AdaGain rather than AdaGrad | Online continual prediction · Stateless tracking problem | MSE | Improved |
| Use AdaGain rather than SMD | Online continual prediction · Rosenbrock function | optimization performance | Improved |
| Use AdaGain rather than AMSGrad | Online continual prediction · Mobile robot sensor predictions | adaptation speed | Improved |
Closest papers by meaning
- Continuous-Time Meta-Learning with Forward Mode Differentiation (ICLR 2022 · similarity 0.88)
- Meta Continual Learning Revisited: Implicitly Enhancing Online Hessian Approximation via Variance Reduction (ICLR 2024 · similarity 0.88)
- Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent (ICML 2025 · similarity 0.87)
- An Effective Dynamic Gradient Calibration Method for Continual Learning (ICML 2024 · similarity 0.86)
- Learning to Continually Learn with the Bayesian Principle (ICML 2024 · similarity 0.86)
- Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
