Research map / Optimization / Nonconvex and smooth optimization
Quantitative Convergences of Lie Group Momentum Optimizers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes two momentum optimizers on compact Lie groups (Lie Heavy-Ball, a splitting scheme, and a new Lie NAG-SC) using only gradient oracle and exponential map, proving Heavy-Ball lacks acceleration while Lie NAG-SC achieves square-root condition number dependence under local strong convexity.
Problem
Convergence rates of momentum-based optimization on Lie groups, where existing accelerated manifold methods need costly logarithm maps or parallel transport.
Approach
Left-trivialized momentum turns Lie group momentum dynamics into Euclidean-like updates; a splitting discretization (Lie Heavy-Ball) is shown to be non-accelerated, and a new Lie NAG-SC discretization with a gradient-difference correction term achieves sqrt(kappa) dependence using only exp map and gradients.
Key results
- Lie Heavy-Ball attains rate (1+mu/(16L))^{-1}, linear condition number dependence, i.e., no acceleration over Lie GD.
- Lie NAG-SC attains rate (1+(1/30)sqrt(mu)*min{1/sqrt(2L), 1/(2p(a))})^{-1}, giving sqrt(kappa) dependence with a curvature-dependent factor.
- Numerical SO(10) eigendecomposition experiments show Lie NAG-SC converges much faster than Lie Heavy-Ball, especially for ill-conditioned problems.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the gradient-difference term to Lie Heavy-Ball's momentum update (ablation) | Lie Heavy-Ball · Locally geodesically strongly convex Lie-group objectives | convergence rate: \left(1+\frac{\mu}{16L}\right)^{-1} -> \left(1+\frac{1}{30}\sqrt{\mu}\min\left\{\frac{1}{\sqrt{2L}}, \frac{1}{2p(a)}\rig | Helped |
| Use Lie Heavy-Ball momentum instead of momentumless Lie GD | Lie Heavy-Ball · Locally geodesically strongly convex Lie-group objectives | convergence rate | No clear effect |
| Use Lie NAG-SC instead of Lie Heavy-Ball | Lie-group optimization · Eigenvalue decomposition on SO(n) | convergence speed | Helped |
Surprising findings
- Lie NAG-SC's rate depends on a curvature-related factor 1/p(a), so acceleration is weaker than the Euclidean sqrt(mu/L) rate; authors are unsure if this is proof-technique or intrinsic.
- No monotone modified energy is established for Lie NAG-SC; only the Lyapunov function is shown to decrease, unlike Lie Heavy-Ball.
Limitations
- Analysis is local: requires local geodesic strong convexity and an initialization in a sublevel set; global convergence rate not provided for NAG-SC.
- Restricted to compact Lie groups with left-invariant metric where ad is skew-adjoint.
- Experiments limited to one problem (SO(10) eigendecomposition) with n=10.
Experimental setup
Benchmarks: eigenvalue decomposition on SO(10) with synthetic symmetric matrices
Metrics: potential value U(g_k)-U(g*) Lyapunov function value
Baselines: Lie GD (Riemannian GD on Lie groups) Lie Heavy-Ball
Builds on: Nesterov accelerated gradient (strongly convex, NAG-SC) Polyak Heavy-Ball Riemannian gradient descent variational optimization on Lie groups
Closest papers by meaning
- Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms (ICML 2024 · similarity 0.84)
- Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization (AAAI 2025 · similarity 0.84)
- Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings (NeurIPS 2025 · similarity 0.83)
- A geometric framework for momentum-based optimizers for low-rank training (NeurIPS 2025 · similarity 0.83)
- Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups (ICLR 2025 · similarity 0.83)
- Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.