Research map / Optimization / Nonconvex and smooth optimization
SERENA: A Unified Stochastic Recursive Variance Reduced Gradient Framework for Riemannian Non-Convex Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SRVRG, a unified stochastic recursive gradient estimator (SRGE) and Riemannian framework SERENA with R-SRVRG and R-SVRRM methods achieving optimal IFO complexity for Riemannian non-convex optimization under retraction and vector transport.
Problem
Stochastic non-convex optimization on Riemannian manifolds (finite-sum and online) where variance reduction methods lack a unified formulation and theory, and hybrid estimators have not been developed for Riemannian settings.
Approach
Combines SVRG-type and SARAH-type estimators into a recursive hybrid estimator (SRVRG), defines a unified stochastic recursive gradient estimator (SRGE) that subsumes SGD, SVRG, SARAH, STORM, SVRRM and Hybrid-SGD, and extends it to Riemannian manifolds via retraction and vector transport with a single convergence theorem.
Key results
- R-SRVRG and R-SVRRM achieve IFO complexity O(min{n^{1/2}ε^{-2}, ε^{-3}}) in finite-sum and O(ε^{-3}) in online settings, matching the lower bound.
- Experiments on PCA, LRMC and Riemannian centroid tasks show R-SRVRG and R-SVRRM outperform the compared Riemannian methods in optimality gap / test MSE versus IFO calls.
- R-SRVRG shows significantly better performance than R-Hybrid-SGD in the reported experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use R-SRVRG instead of other compared methods | PCA on MNIST · MNIST | optimality gap | Helped |
| Use R-SRVRG instead of other compared methods | PCA on ijcnn1 · ijcnn1 | optimality gap | Helped |
| Use R-SRVRG or R-SVRRM instead of other compared methods | LRMC · Movielens-1M | test MSE | Helped |
| Use R-SRVRG or R-SVRRM instead of other compared methods | LRMC · Jester | test MSE | Helped |
| Use R-SRVRG or R-SVRRM instead of other compared methods | Riemannian centroid computation · Extended Yale B | optimality gap | Helped |
| Use R-SRVRG or R-SVRRM instead of other compared methods | Riemannian centroid computation · Kylberg | optimality gap | Helped |
| Use R-SRVRG instead of R-Hybrid-SGD | Riemannian stochastic optimization · PCA, LRMC, and Riemannian centroid tasks | performance | Helped |
| Use R-Hybrid-SGD instead of R-SRM | Riemannian stochastic optimization · PCA tasks | performance | Helped |
Limitations
- Framework is limited to smooth problems
- Assumes isometric vector transport and several standard assumptions (bounded gradients, neighbourhood conditions)
Experimental setup
Datasets: MNIST ijcnn1 Extended Yale B Kylberg MovieLens-1M Jester
Benchmarks: PCA on MNIST PCA on ijcnn1 LRMC on MovieLens-1M LRMC on Jester RC on Extended Yale B RC on Kylberg
Metrics: optimality gap test MSE IFO complexity
Baselines: R-SD R-CG R-SGD R-SRM R-SPIDER R-AbaSRG R-AbaSVRG R-PAGE R-Hybrid-SGD
Builds on: STORM Hybrid-SGD SVRRM R-SVRG R-SRG R-SPIDER
Closest papers by meaning
- Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization (AAAI 2025 · similarity 0.91)
- Finite-Time Analysis of Stochastic Nonconvex Nonsmooth Optimization on the Riemannian Manifolds (NeurIPS 2025 · similarity 0.89)
- Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds (ICML 2024 · similarity 0.88)
- Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method (ICML 2025 · similarity 0.88)
- Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms (ICML 2024 · similarity 0.88)
- Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.